More Gentoo development links than you probably need

Since I've been less active packaging for Gentoo recently, I've been trying to at least lurk in #gentoo-dev-help and give a helping hand with packaging. If I can't help them directly, at least I try to give them a gentle shove towards something that can.

As a result, I've been collecting various bits of documentation about how to package stuff. This post is a result of this, as well as some other digging around I did to flesh it out.

General development documentation

Channels of communication

Lots of projects and herds within Gentoo have their own mailing lists and irc channels.

Language-specific

Packages written in different languages often have their own way of dealing with packaging:

Herds

Same goes for herds.

Other stuff

These don't fall into any other categories, but are still useful.

Permalink Edit Destroy

Better partials with better-partials

The other week, Dan Croak pointed out better-partials to me. Hadn't gotten a chance to play it until now.

The gist of better-partials is that it's a syntactical sugar for render :partial.

Here's a typical haml snippet using a partial:

- form_for(@video) do |form|
  = render :partial => '/videos/form', :locals => { :form => form }

With better-partials, we can refactor it to:

- form_for(@video) do |form|
  = partial 'videos/form', :form => form

This saves a little typing, and makes it a bit more clearer:

  • You just say you want a partial, rather than to render a partial
  • Automatically passes in options to :locals (with a few special exceptions)

There are a few other usages of better-partials, but I haven't gotten to use them yet.

It's not that big of a deal, but it seems like a nice little improvement to me.

Permalink Edit Destroy

Using a Gentoo Prefixed shell as your login shell

One thing that kind of annoyed me about Gentoo Prefix is that you always needed to do a little work to enter the prefix:

$ ./bootstrap-prefix.sh /path/to/prefix startscript

Personally, I just want to always live in a prefixed shell. Amazingly enough, this is really easy to do.

  1. First open up /etc/shells as root
  2. Add a line like /path/to/prefix/bin/bash and save
  3. As your user, run: chsh -s /path/to/prefix/bin/bash
  4. Enter your password
  5. Done

Now, everytime you open a terminal, it will be running a bash gloriously built by Gentoo

Permalink Edit Destroy

Boston.rb Hackfest 5/07 Post Mortem

By now, I think it's a running joke that we'd work on the recommendable plugin we started a month or so ago.

Instead, we worked on the Factory stuff. This article captures most of the ideas of using factories for testing and reasons for using them instead of fixtures.

The crew tonight:

Overview

As the article mentions, someone did implement a plugin which captures most of these ideas.

There were a few reasons (and some more which I don't remember) why we didn't like this plugin.

  • Plugin, not a gem, so unusable outside of rails
  • Pollutes the namespace of Object
  • No tests

So, the journey for better Factories begins. The basic ideas/goals we settled on is this:

  • Define your factories in test/test_helper.rb
  • Be able to define a factory for a class with default options
  • Be able to define alternate factories for a class, for example, to create an admin user versus a normal user
  • Flexibility for defining default attributes, ie be able to call other methods
  • Choice of making new instance, persisting a new instance, stubbing an instance, or getting valid attributes
  • Flexible API for actually calling the factories

The API

So we started out with a basic API that Dan Croak had outlined:

Factory.define do |factory|
  factory.add :project do |project|
    project.name {"factoryfu"}
    project.homepage_url {"http://factoryfu.rubyforge.org"}
  end

  factory.add :event, :meeting do |event|
    event.event_type {"Meeting"}
    event.date {Time.now}
    event.description {"an sick meeting"}
  end

  factory.add :event, :hackfest do |event|
    event.event_type {"Hackfest"}
    event.date {Time.now}
    event.description {"a cool hackfest"}
  end
end

As for usage of the factories, we had 3 possibilities:

Factory.create :project, :name => 'ambitious_sphinx'
Factory.create_project :name => 'ambitious_sphinx'
create_project :name => 'ambitious_sphinx'

I'm not going to say much about these, aside from the first being the simplest to implement, the second being the most explicit about what you are doing, and the last is the most concise.

Because it seemed like the simplest, we started implementing the first. I believe that the other two syntaxes can actually be emulated using it.

To simulate the second, as we register factories, we can create the appropriate methods, ie create_event, new_event, valid_event_attributes.

To simulate the third, we can build on the second, and probably do some magic to delegate from TestCase to our Factory.

Implementing it

For simplicity's sake, we started with our work from last week's hackfest on bostonrb.org.

We added the snippet for defining factories into our test_helper.rb, made a new model unit test which used the factory methods, and started a Factory class.

We basically ran the test, watched it blow up horribly, and added some code, until eventually the test passed.

You can see the work we ended up in the hackfest repository under 'boston_rb'

Of particular interest are:

So, I'm just go over a few of the more interesting bits of the implementation:

Factory.define

This is the entrance into defining our factories.

The implementation isn't very interesting. We just call the block on Factory. We were originally making new instances of Factory, and passing that to the block, but we then realized that this made it a little trickier to have factory methods be class-level methods.

Factory.add

This is what you use to define how you the factory behaves.

It takes a block which defines how to create new attributes. If you look at the snippet:

event.event_type {"Hackfest"}
event.date {Time.now}

Notice that event.event_type and event.date are actually taking blocks. This allows us to do Time.now, and have this not actually be invoked until object creation.

Factory.add actually passes your block a magical Hash which captures this attribute/attribute creation data. We'll talk about this has in a little bit.

After we get back from the block, we store in @@model_templates, keyed by the model and kind of the factory.

The Factory methods

We provide three factory methods:

  • valid_attributes: a Hash of valid attributes for the object
  • new: a new unpersisted instance, using valid_attributes
  • create: a persisted instance, using new

You invoke them like:

Factory.valid_attributes :project
Factory.new :project, :name => 'factoryfu', :homepage_url => 'http://somewhere.com'
Factory.create :event, :hackfest, :date => 3.days.ago, :description => 'your looking at it'

The way we parse the arguments is a little tricky. Initially, we only supported the first 2 lines. This meant our method signature looked like:

def self.new(model, attributes = {})

When we went to support the 3rd line, we thought to go like:

def self.new(model, kind = 'default', attributes = {})

By doing so, you break the 2nd line, because kind would get set to our attributes.

activesupport provides a solution: Array#extract_options. Basically, it looks at the Array, and pops the last element off the end if it's a Hash. We use it like:

def self.valid_attributes(model, *args)
  kind, attributes = parse_args(*args)
  # ...
end

protected

def self.parse_args(*args)
  attributes = args.extract_options!
  kind = args.first || 'default'
  [kind, attributes]
end

BlockfulHash

We made a subclass of Hash which lets you define keys like:

event.date {Time.now}
event.description {"This is going to be SICK!"}

Such that it's equivalent to:

event[:date] = {Time.now}
event[:description] = {"This is going to be SICK!"}

Here's the implementation:

class BlockfulHash < Hash
  def method_missing(method, *args)    
    # When Proc.new is given no arguments,
    # it returns a Proc for the block passed to the current method
    # Proc.new... yeah, it's officially a hackfest.
    self[method.to_sym] = Proc.new
  end
end

This BlockfulHash is what gets passed to the block given to Factory.add.

What's next

So, we have something workable here:

  • Uses the syntax we laid out for defining factory
  • Supports the first syntax for using factories (Factory.create :event)

As far as what is left to do:

  • Extract it out of the boston_rb code base
  • More tests
  • Gemify
  • Support Factory.create_event syntax
  • ...
  • PROFIT!!!

Addendum

Some links and other ideas tossed around over the course of the night:

Permalink Edit Destroy

Managing RubyGems on Gentoo

For Ruby on Gentoo, it has been asked how one should go about managing RubyGems. You have a few options:

  • Use Gentoo packages exclusively
  • Use gem exclusively
  • Mix and match and pray

Mix and match

Offhand, mixing and matching seems like a bad idea.

Doing so, you might install a gem using portage, but then you could use gem update to update it. Then you update it with portage...

Overall, seems like it could be problematic in the long run.

Gentoo packages exclusively

Being a Gentoo packager, my first take is that using packages is the way to go.

Some pluses:

  • One stop for installing/updating packages, ruby or otherwise
  • For gems with native extensions, can have dependencies on the things the native extensions use

Of course, there are a number of disadvantages.

  • There are tons of gems out there
  • Some update frequently
  • Lot of packages to be keeping up with stable keywords
  • Despite gem ebuilds being pretty straightforward, there is a bit of grunt work involved on to verify dependencies are accurate
  • Lots of gems are unpackaged, because they are usually packaged as requested or as developers need them
  • Despite being able to install multiple gems at once, packages usually aren't slotted that way

Gems exclusively

If you're a Ruby developer, the most natural way would be using RubyGem directly. Some nice things include:

  • Most direct way to have the latest and greatest gems
  • Can have as many versions installed as you want
  • Zero maintenance for Gentoo packagers (sweet!)

Of course, not without their drawbacks:

  • If a gem uses native extensions, it can be tricky to determine what the it's dependencies are
  • Problematic when other (non-ruby) packages need to depend on gems

My current practices

So where does that bring us?

Here are the practices I've taken to using for my Ruby and Rails development and deployment needs.

In development

On my development machine (4 MacBook running Leopard with Gentoo/Prefix), I've using RubyGems directly.

If I'm working on a Rails app, and it needs any gems, I've taken to vendoring everything. This type of thing will be supported out of the box for Rails 2.1.

Of course, this wouldn't work so well for things with native extensions, like hpricot.

In production

I avoid installing any gems in production as much as possible. If an app needs something, it should be vendored, like I just described.

For gems needed by my apps that have native extensions, or are otherwise needed in production (like rake and the mysql gem), again, I'm using RubyGem directly.

Permalink Edit Destroy

Boston.rb Hackfest 4/29 Post Mortem

So, we were just kidding about working on the recommendable plugin. We actually worked on the bostonrb.org website some more.

The crew tonight was:

The significant things we worked on were:

  • Projects have descriptions now
  • Events now have a location
  • Reorganized events page
    • left side has all upcoming events
    • center has next event with google map
    • right side has nothing now, but we're thinking to put a calendar type view

Some articles, resources, that came up while hacking away:

The fruits of our labor have been pushed to http://bostonrb.org, and the source can be found at https://svn.thoughtbot.com/hackfest/boston_rb.

Permalink Edit Destroy

piston and git for the win

Piston has had support for git (both importing from, and into) for a little over a month.

Before that, I felt like developing a rails app in git was a bit painful. Considering most plugins are kept in subversion, you're only real option is using svn export to install plugins, and check that into git. There was braid, but I really didn't have any luck with it.

But, like I said, it's all good now. It hasn't been officially released (ie as gems), but you can build and install it yourself easily enough.

$ git clone git://github.com/francois/piston.git
$ sudo gem install -y main open4 log4r
$ cd piston
$ sudo rake install_gem

Pretty straight forward to use.

$ cd vendor/plugins
$ piston import git://github.com/technoweenie/restful-authentication.git
INFO main: Guessing the repository type
INFO main: Guessing the working copy type
INFO main: Checking out the repository
INFO main: Copying from Piston::Revision(git://github.com/technoweenie/restful-authentication.git@42083ffa31e0a9792472780854ddd81bcc9b2f61)
INFO main: Checked out "git://github.com/technoweenie/restful-authentication.git" 42083ff to "restful-authentication"

And since it's since it's being developed in a git repository, and on GitHub, it's extremely easy to start contributing.

I mean, if I can contribute something, I'm sure you can pull it off too :)

Permalink Edit Destroy

Boston.rb Hackfest 4/15 Post Mortem

Last Tuesday, we had a small, but dedicated showing at the Hackfest:

Going in, we didn't really have a something in mind to work.

We thought to pick up the app we started in merb for organizing katas. But when it came down to it, we figured it'd be a lot quicker if we just did it rails.

We actually wanted something a little more general, kind of a one stop shop for Boston Ruby stuff. Here's what we wanted:

  • Keep track of events (meetings, hackfests, and katas)
  • Projects that have come out of above-mentioned meetings
  • Recent commits from these projects

Tools we used:

You can see the code we ended up with in the hackfest repository.

Or, you can see it in action.

For simplicity's sake, we didn't initially bother with users/permissions/etc, so be gentle with it.

Permalink Edit Destroy

How to raise the bar to contribute to your open source project

I've been doing quite a bit of Ruby development lately, not going to lie.

There have been several times where I'm working with something, and need a better understanding of how it works. Or maybe I think I have a bug. Or maybe I want to add some sort of improvement.

But, it's not always so easy to do so. So, if you want to raise the barrier to contribute to your project, here are some practices for you:

  • Do...
    • Make a single blog post the authoritative home page
    • Distribute tarballs that can't reproduce your artifacts (ie gems)
    • Use RubyForge for things like ticket tracking
  • Don't...
    • Don't make it obvious where to find the source under version control.
    • Don't make it obvious where to file bugs
      • Most projects that use their RDoc as their web page
    • Write RDoc, or otherwise comment your code
    • Tag releases

Certainly, this some of this list doesn't just apply can apply to many other projects, not just Ruby ones.

Permalink Edit Destroy

Dead easy nginx configuration on Gentoo

Considering the hype about nginx in the Rails community, I figured it was about time to try it out.

Being the slacker that I am, I didn't really want to have to learn how to use it. Fortunately, Ezra gives us a good starting point for a configuration.

Then Chris Wanstrath comes along and creates nginxconfiggenerator to make it even easier, by providing a generator around this configuration.

One kink though. Running on Gentoo and using the latest nginx package. Default template has paths not suitable for this. Bummer.

Being the clever fellow he is, Chris's generator does provide a way around this, by letting you specify your own template to use.

I took a little time to tweak his original template for Gentoo consumption. Behold!

Here's how you'd use it, assuming you've gone through the steps already mentioned in the post.

$ wget http://p.caboo.se/182778.txt -O gentoo_nginx.conf.erb
$ generate_nginx_config config.yml nginx.conf \
  --template gentoo_nginx.conf.erb

You should be all set now.

One last note, you need to set fair: false for your sites. The fair directive doesn't seem to work with the current version we have on Gentoo. I'm not positive, but I think it's related to this post by Ezra

Permalink Edit Destroy