Some Huck Hacking

I used to work on a big search product at AOL and still love search, even though that’s not what I do anymore. So, when I saw that IndexTank and Heroku were having a contest to build a cool app with IndexTank’s search-in-a-box, I couldn’t resist. I knew I had to keep it simple since I don’t have a lot of time for hacking outside of work, but I knew I had to do something.

I had two ideas, and went with the simpler one: What would happen if you broke a book down into individual sentences and made it searchable? Would it be useful at all? I decided to try Huckleberry Finn by Mark Twain, since it’s not too long, is public domain, is quotable and full of vernacular that can screw up indexers, and I knew it was available from Project Gutenberg.

I grabbed the text file, cut and pasted each chapter into individual text files and then wrote a Ruby parser to split it up into paragraphs and sentences, which were then written to javascript files. After that was done, I wrapped it in a simple Rails app to display each chapter and paragraph, and then fired all the sentences at IndexTank.

I call the result… Huck Smash, and I think it’s pretty cool.

It was a lot of fun to write an app without a database or ORM, just a bunch of javascript files that Ruby can read and an extremely limited scope. I know it probably won’t win, but it was a lot of fun to write and only took a few hours to put together. Writing the text parser was a lot of fun, and figuring out how to navigate the book and build out the HTML so you can link to an individual sentence was cool.

I’m going to try to spend more time outside of work playing with single-purpose sites and fixing Ficly up. I need to keep things constrained so I don’t bite off more than I can chew or over-commit, but this was so much fun I want to do it again.

I’d love to hear what you think of Huck and any ideas you have for improvements.

Thank You, Open Source!

Thank you
“Thank You” by Darwin Bell

I read Zed Shaw’s blog post on the decline of open source participation last week and it got me thinking about just how much open source software we use at work and how we’re (mostly not) giving back to those communities. So, here’s the first step in me becoming more involved and giving something back, even if it’s just a huge “thank you”. I am trying to be more involved, especially in the MongoDB and MongoMapper communities. I’m probably not going to be contributing code to either, but I’m fairly active on the mailing lists, have reported bugs and am committed to help with the MongoMapper documentation project.

Excuses aside, here’s a list of the big open source things we use on a daily basis and why we love them:

  • Apache – The webserver that holds everything together. It’s used by most of the web, and we use it too.
  • Ruby on Rails – Rails lets us do more faster. We also use a bunch of gems that I’ll list later on.
  • Sinatra – When you don’t need everything that Rails has (a simple API, for example), then Sinatra is perfect.
  • Passenger – Deploying Rails apps used to be a pain. Not anymore! Thank you, Passenger!
  • MySQL – Need an RDBMS? Well, we use this one. And it works pretty darned well.
  • Memcached – We cache everything we can, and memcached helps us do that.
  • MongoDB – We use it because it’s web scale! (if you get that joke, then you’re in the club!) Seriously, we first started using MongoDB just to collect our stats because we’d maxed our poor MySQL instance. Then, I looked deeper and realized it’s perfect for the big top secret thing I’m working on now. Atomic updates and super-fast inserts make it perfect for collecting a lot of data quickly. And it’s now slouch on the query side either. There’s also a great community behind MongoDB. The updates and improvements are frequent and the community is always willing to jump in and help.\
    It’s a nice hybrid between the new school document store databases and a traditional RDBMS.
  • Beanstalkd – A super-fast queue server. It just works, which is why I love it. We queue everything we can. Why? It’s a great way to meter load. If you can only handle 3 jobs running at once, then you only run 3 workers. If you can handle more, you run more. It’s great!
  • And of course, all of our servers are Linux and run hundreds of open source packages that I don’t even worry about.\
    Since our strength lies in Ruby, we try to do everything in Ruby that makes sense. I’m not going to list all the gems we use, but here are a few of my favorites – the ones that make life easier and make programming all day more fun.
  • MongoMapper – Makes working with MongoDB even more fun. I’m on the MongoMapper mailing list, and it’s one of the most supportive and helpful communities I’ve been a part of. It makes using it more fun.
  • memcache-client & beanstalk-clent – they’re how we talk to memcached and beanstalkd
  • hashie – Allows you to very easily create classes built around hashes. Great for wrapping around API’s.
  • typhoeus – My favorite of the many HTTP clients for Ruby.
  • will_paginate – Now I don’t need to do all the horrible gymnastics needs to add “previous” and “next” links to things! THANK YOU!
  • hpricot – My favorite way to parse HTML – with CSS selectors!
  • aws-s3 – A great interface to Amazon S3 (where we store a bunch of stuff)

There you go. That’s pretty much our entire stack. I left out a bunch of gems – most of them we don’t use directly – or that just provide one or two things.

So, thank you to all of the creators and contributors to open source projects out there, especially the ones we use to make our work easier. The web would be a much smaller place if there weren’t dedicated geniuses out there making this stuff, and the world would be a poorer place for it. I promise to be a better member of the community and contribute where I can!

Murray Wilson Is Awesome

My pal Murray Wilson does great things – he and AWOL take kids the system doesn’t want and teaches them to take apart, clean and refurbish computers the system doesn’t want – computers that would otherwise go to the landfill.

They then put linux on them and put them out into the community with families that need them. He’s one my absolute favorite people in Savannah (nay, the world) and I’m proud to know him.

The computers will, of course, end up in the landfill eventually, but the “Goon Squad” gives them easily another 2-5 years of life, and the kids learn useful and marketable skills. It’s a win-win, and an amazing program and Murray and AWOL built from the ground up.

If you can spare it, AWOL can always use some help. Every little bit helps, and every kid they help is one that’s not in the juvenile justice system or out on the street by themselves.

Murray is awesome in the best sense of the word.

Thoughts on Kanban

I was introduced to Kanban by Brad Taylor from Rails Machine when we were talking about agile and how to do it in an ops environment where there’s a lot of reacting that’s hard to plan for. Since I come from the product dev world, where releases are easier to plan, Scrum works pretty well.

What’s Kanban? The best introduction I’ve read so far is Kanban Developmnt Oversimplified. I also attended a great introduction to Kanban by David Laribee at BizConf, so some of my thoughts came out of that intro and not the article.

In my own words, Kanban is an agile approach to development where you take away most of the structure found in Scrum and replace it with visual queues for progress. You basically have buckets for things: Planning, Defined (ready for development), Being Developed, Testing and Ready. You place limits on how many things can be in those buckets, which keeps you from working on too many things at once. The visual queues are displayed on a board with columns for each bucket. That allows you, in a small team, to see where your bottlenecks are and where you have availability. I think it’s a great approach for a team of equals, where you don’t really need the metrics you get from Scrum (team velocity, estimates vs. actuals, etc), or a small centrally located team. The physical Kanban board becomes useless when you’ve got remove people, since they can’t act on their own cards or see the board all the time.

If I was running a 3 person shop where we all worked on the same product, I’d do Kanban instead of Scrum. I also think the visual representation of the board – what’s being worked on right now – is a great tool for seeing your team’s current status. We’re trying out Wallsome as our Kanban board (we can’t do the physical board because we have folks in Spain). I chose it because it uses our existing Basecamp data, which makes the adoption cost relatively low for us. I checked out a bunch of other web-based Kanban tools and they all required me to re-enter everything, which I just don’t have time to do.

But, I’m running a team where I need to help people improve. So, having the data I get out of Scrum is extremely important – and Scrum just works for us. We know what we’re working on, when the next release to production is, and can easily communicate that to whoever needs to know. Switching to Kanban would only take things away without improving efficiency (OK, it might, but I’m not seeing how right now). We are going to use Wallsome for a while and see if it helps us keep better track of the flow of tasks during a sprint, but I don’t see us switching to Kanban completely anytime soon.

So, I think Kanban’s great for teams where you spend a lot of time reacting to events outside of your control, small centrally located teams or teams where they’re just starting to get into Agile and don’t have a lot of different projects. But, our team has been doing Scrum for two years, and it works for us. We’re already Agile and have a well-established process (and more importantly, a change management process for those things we just have to “react” to on short notice) with Scrum that works for us. The switching costs at this point are higher than I’m willing to bear.

Rails Resources

Shawn Medero asked for some Rails resources on Twitter last night, and here they are. All of the blogs are from my feed reader and the links are things that are pretty much always on one of my first five browser tabs. I’m sure I’m missing some great Ruby and Rails blogs from this list, so if you have any… bring ’em on.\
Documentation:

Blogs:

Books:

  • I use O’Reilly’s Ruby in a Nutshell as a desk reference all the time. It’s so worn out, I probably need to order another copy (although I do have the PDF now).
  • With Rails 3 just around the corner, I would wait if you want to get a Rails book. Maybe start with the Manning Early Access Program and get Rails 3 In Action

I only listen to one Ruby podcast: Ruby5 – It’s only a couple times a week and only 5 minutes at a time, which is perfect for short attention spans.

10 Years

I’ve been blogging, right here, for ten years. The first post on lawver.net was on 07/20/2000 and didn’t say a whole lot. Since then, though, Jen and I have posted 2,631 entries. Jen didn’t start blogging here until about 2005, so out of that, almost 2,000 of them are mine.

I don’t think I’ve stuck with a hobby longer than this, except maybe collecting comic books when I was a kid (non-stop from age eleven until 22 and then off and on collections).

I’m pretty proud of this little blog. Yes, the design is old, and it’s slow at times. But, it’s a record of our lives over the last ten years that I wouldn’t have otherwise.

Here’s to the next ten and whatever comes next.

Today’s Idea: TED Lesson Plans

While we were up in DC for Steve’s wedding, TEDx Creative Coast happened down here in Savannah. I’ve had several conversations with folks about how high energy it was and how excited everyone was about the event, and that there has to be some way to keep that energy going.

I got to thinking about it, and how cool would it be to create lesson plans for teachers around TED talks? They could show the video, then have a discussion with the kids, or some activity around the topics discussed in the video.

I don’t have time to implement it, but I’m putting the idea out there in case someone wants to run with it. I figure it could be a wiki, or something wiki-like, where you take the video, and then build the lesson plan below it.

I’m not sure TED has an API for pulling all the talks, but it wouldn’t be too hard to scrape the podcast feed and do it auto-magically.

If you do something like this, or know of something that already exists, let me know!

My Summer Plan – Produce More, Consume Less

When I get home from work, I’m usually pretty tired (OK, most nights, I’m really tired). So, I don’t do much other than:

  • Sometimes make dinner. Usually Jen makes it or we order out, but I do cook on occasion.
  • Play video games with the kids
  • Watch TV
  • Screw around on my ubuntu laptop, but mostly I surf or play games while watching TV.

And that’s it. On rare occasions, I do community stuff like go to Refresh.

Since I don’t travel much anymore, I can’t remember the last time I read a book. I have a stack of things I want to read that I just haven’t gotten around to. I have a ton of little personal projects that are sitting around partially done that I should finish.\
So, this summer, I’m going to watch less TV and do more. I’m going to read some books, write some code and have some fun. I know reading isn’t “producing” anything, but it’s more educational and intellectually stimulating than watching fictional characters on TV have and resolve fictional problems.

My first project? Redo this blog. I realized we’ve been running this blog Movable Type for almost 8 years, and running it on Dreamhost for over 7. This “design” (if you can call it that) has been here, basically untouched, but over 5. It’s time to do something else. I rarely post anymore because it’s slow. I put most little stuff over on tumblr, and only come here for longer posts.

My plan for the blog is to build my own from scratch. I know, I know, the world has enough blogging platforms. I’m not going to make a “platform”. It’s just a blog for Jen and me that’s fun to play with and easy to maintain. It’ll be on Rails and backed by MongoDB, so it should be trivial to add new stuff, rip it out, start over, or move it around.

I plan on taking my time, playing with new stuff along the way (web fonts, HTML5, writing tests, the design, etc), and will roll it out when I feel like it.

I’ll take breaks to do some reading, but won’t be doing any other personal hackery until it’s done. Once it’s finished, if there’s time, I have an idea for a twitter app I’ve been wanting to play with and couple things I’ve started with other folks that I want to finish up.

It should be fun. I’ll post updates as I feel like it. In the meantime, what are you going to produce this summer?

Enabling Creativity

Max and I presented today at the Telfair’s Pulse art and technology fair for “family day”. Our presentation went over some of the fun stuff we’ve done together with technology and how people can get their kids to find creative outlets through geekery. We had a lot of fun, and Max had a blast presenting (he did a great job).\
Here’s how we came up with the presentation:

* Two weeks before the presentation, Max and I sat down and talked about what stuff we’ve played with he’d want to tell people about and built an outline.

* I went back later and created the actual presentation, filling in the gaps and figuring out what I wanted to say in the intro and conclusion.

* The week before the presentation, we practiced the demos and came up with the idea of having him tell me how to do everything (instead of the traditional “Mr. Wizard” style approach).

* The morning of, we went through it again and made sure he had some idea of what he wanted to say during his parts.\
I didn’t want to put a huge burden on him, since this was his first time in front of what could have been a large crowd. But, he was very involved in writing the presentation and walked me through all the demos.\
He had so much fun that he wants to present again, maybe at this year’s Geekend.\
Here are the slides if you’re interested:

Weird Dreams

I had a weird dream last night (two, actually, but I’m only concerned about writing down the first one). In the dream, I was back at AOL sitting through a horrible product requirements meeting when I lost it and started yelling about how bad the requirements were, how they didn’t do anything original, were a waste of paper and no one would use this thing even if we built it (I don’t even remember what it was now). I got in a fight with the product manager, and all I remember of the screaming match was that she said something like, “You’re not the only ship on this sea, pal,” to which I replied… and I remember me screaming it: “Not the only ship?! I’m the sea!“\
Then, I got fired. It was a strange experience, watching dream me pack up his crap in boxes and get escorted out. I lost it a few times in my thirteen years at AOL (wait, sorry, now it’s “Aol.”), and one or two of them almost got me fired, but those were early on when I was still in tech support. I lost it in meetings a handful of times (which I think is a pretty good record considering how many awful product meetings I sat through) and called BS where I needed to, but I don’t think any of them ever got me close to the “terminating offense” line.\
Yeah, I don’t know what it means either, but I thought it was a pretty good comeback, especially for a dream.\
The other one was a nightmare where I was Doctor Who. It was so scary, I actually woke up and had a hard time getting back to sleep. This robotic zombie fell on me, had me pinned to the floor and kept saying “I know what you are” over and over again. shudder.