April 24th, 2013

Programming Bits #2

Coffee-time sized bites of eccentric programming knowledge, served lukewarm. An emphasis on modern languages like Ruby and SmallTalk with flashbacks to the tried-and-true.

Web Development

Podcasts and Screencasts for Extended Learning

http://www.smashingmagazine.com/2013/04/19/podcasts-for-designers-developers

An awesome collection by Smashing Magazine that contains a tremendous number of valuable podcasts and screencasts for learning more about web development and the business of web development.

Bootstrap WSYIWYG

http://www.layoutit.com/

What you see is what you get, bootstrapped. This tool allows you to mockup an app based on bootstrap and generate the layout. It does not build your app for you, but gives you the initial visual layout framework based on bootstrapped elements. It might be a nice way to quickly scaffold up the HTML/CSS required for a bootstrap-based site, so you can get back to focusing on your app’s core logic. Though, it likely doesn’t understand a web framework, such as Rails, so you’ll have to convert it’s output by hand.

HTML/CSS Goodies

NetFlix is Moving From Silverlight to HTML5

http://techblog.netflix.com/2013/04/html5-video-at-netflix.html

What this means is that HTML5 video is becoming the most common way to present video on the web. It helps to mark the death of Silverlight and Flash for playing videos on web pages.

LinkedIn Dumped HTML5 in Favor of Native Apps

http://venturebeat.com/2013/04/17/linkedin-mobile-web-breakup

Conversely, LinkedIn decided to dump HTML5 in favor of creating native apps. While not related to video itself, it points out that HTML5 is not necessarily the language/platform of choice for making mobile applications, and large successful companies like LinkedIn are seeing benefits of creating native apps over mobile-friendly HTML apps.

Reduce Redundant CSS

http://flippinawesome.org/2013/04/22/automating-csscss-using-grunt

Using the CSSCSS gem, you can analyze your existing CSS and find code duplication. It helps to identify areas of your CSS (like you might do for ruby using a smell detector like CodeClimate) that can be DRYed up. This article discusses using Grunt to automate this a bit (in vein of guard and other change-aware automation tools).

Heroku, Heroku, Heroku

Heroku Now in Europe

https://blog.heroku.com/archives/2013/4/10/europe-region

Heroku has released their Europe region. This allows Heroku apps to run in Europe, so that they are closer to European customers and your apps can perform more expedient in that region. It removes the mandatory 80-100ms latency increase caused by crossing the international backbones, amongst other things.

“The physical proximity of the Europe region to European end-users means reduced latency, often resulting in a dramatic improvement in app responsiveness to those users. We’ve observed performance improvements of 100ms per request or more for European end-users.”

Migrating an App to the Europe Region

http://vimeo.com/64197387

Heroku has created a screencast that shows just how simple it is to move an app to their Europe region.

Design

Ten Principles for Good Design

https://www.vitsoe.com/eu/about/good-design

Not necessarily web design focused, but presents excellent design fundamentals that can serve to enhance the design process.

Creating Responsive Email Templates

http://www.zurb.com/playground/responsive-email-templates

Just like responsive design is important for the web, it translates to email as well. You can’t control what size your market’s email client is.

Making Stuff Out of Things

http://kadavy.net/blog/posts/stuff-and-things/

A humorously written article that talks about focusing on making stuff over using things. Focus on the making, not the using.

That’s it for Programming Bits #2. If this was useful or helpful to you in anyway, I’d like to hear about it. And please, share it with others if you think there’s something useful there worth sharing. While I do my best to source these relevant bits of news myself, I also rely on you to contribute anything interesting you come across, so send them it for the next Programming Bits!

April 23rd, 2013

Programming Bits #1

Coffee-time sized bites of eccentric programming knowledge, served lukewarm. An emphasis on modern languages like Ruby and SmallTalk with flashbacks to the tried-and-true.

Cargo-cult Programming

http://en.wikipedia.org/wiki/Cargo_cult_programming

Cargo cult programming can also refer to the results of (over-)applying a design pattern or coding style blindly without understanding the reasons behind that design principle in the first place. Examples are adding unnecessary comments to self-explanatory code, adding deletion code for objects that garbage collection would have collected automatically with no problem, and creating factory objects to build simple objects. It often happens when programmers are inexperienced with the programming language, or simply overzealous.
A related term in software engineering is cargo cult software engineering, coined by Steve McConnell.
McConnell describes software development organizations that attempt to emulate more successful development houses, either by slavishly following a software development process without understanding the reasoning behind it, or by attempting to emulate a commitment-oriented development approach (in which software developers devote large amounts of time and energy toward seeing their projects succeed) by mandating the long hours and unpaid overtime, when in successful companies these are side-effects of high motivation and not requirements.
In both cases, McConnell contends that competence ultimately determines whether a project succeeds or fails, regardless of the development approach taken; furthermore, he claims that incompetent “impostor organizations” (which merely imitate the form of successful software development organizations) are in fact engaging in what he calls Cargo cult software engineering.

Powwed Up

Pow
An on-demand DNS and HTTP proxy for your Rack/Rails apps in development. This lets you easily access http://yourapp.dev/ on your local system so that you can easily test your app instead of manually running the server.
Pow for Alfred (similar to Quicksilver)
Powify
Manage your pow instances from the command line. Makes it really easy to see what pows you have, and inject new apps into pow.
Powder
An alternative to powify.

Speeding Up Rails

Zeus
Speeds up your tasks such as console, migrations, and server loading by preloading Rails.
rails-dev-boost
Speeds up Rails in development mode.
turbo-sprockets-rails3
Speeds up asset pipeline precompile by only compiling assets that have changed since last time.

Heroku Plugins

Using Heroku CLI Plugins
Jenkins Plugin
Heroku Accounts
Easily manage your heroku account’s SSH config on your local system.
Papertrail Plugin
Heroku PostgreSQL Extras Plugin
Calculate performance details about your postgres database, such as cache hits and see long running queries.
That’s it for the first Programming Bits. I’m not sure about a release schedule for these yet because I still need to see how effective it will be for me to make them, and just how much interesting content I’m able to come across. I’ll likely just collect interesting things as I come across them, and then release a Programming Bits when I have enough content to feel like it will be a worthy release. If there’s someone you think who might benefit from this, feel free to forward this along.

March 16th, 2011

Project Euler: Problem1/Variation2 – Add all the natural numbers below one thousand that are multiples of 3 or 5

My previous post contained my first variation for solving Problem 1 of Project Euler. A basic loop was intentionally used to illustrate a brute force method of determining the answer. The solution is easy to understand and the code is very simple. For Variation 2, I decided to simply refactor Variation 1 so that it was more reusable and more clear. I’ll explain what was discovered after the code dump:

#!/usr/bin/ruby
#
# Project Euler
# Problem #1 - Add all the natural numbers below one thousand that are multiples of 3 or 5.
# Solution #2 - refactored loop
#
# Copyright 2011, Kevin Elliott, kevin@phunc.com
#

# Definitions
floor = 1
ceiling = 999999
multiples_of = [3, 5]

class DivisibleDetector

  # Multiple of a value?
  def self.multiple?(i, multiple)
    (i % multiple) == 0
  end

  # Multiple of one of the values in an array?
  def self.multiple_of_array?(i, multiples)
    is_multiple = false
    multiples.each do |m|
      if self.multiple?(i, m)
        is_multiple = true 
        break
      end
    end
    is_multiple
  end
  
  def self.find_divisible(start=1, stop=999, multiples=[])
    results = []
    (start..stop).each do |i|
      results << i if self.multiple_of_array?(i, multiples)
    end
    results
  end

  def self.sum(results)
    sum = 0
    results.each { |r| sum += r }
    sum
  end
  
end

# Main
results = DivisibleDetector.find_divisible(floor, ceiling, multiples_of)
puts "Numbers below "+ceiling.to_s+" that are multiples of 3 or 5:"
results.each do |r|
  puts "  "+r.to_s
end
puts "Sum = " + DivisibleDetector.sum(results).to_s

As expected, I noticed that Variation 2 was slightly slower than Variation 1, because it introduces several method calls as well as a new inner loop for checking to see if the iterating value is a multiple of multiple values in an array. While this allows for flexibility (say, for example, you add a few more numbers to check for divisibility by) and modularity, it introduces complexity and a slight performance degradation.

Aside from using a loop as a strategy for solving this problem, the structure and format of Variation 2 is “the Right Way” to structure your code. But remember this comes at a cost of complexity and performance degradation. To most this is a small price to pay to have a consistent structure for organizing your code.

Variation 3 will attack the problem from a whole other perspective, rather than simply refactoring a previous variation.

March 16th, 2011

Project Euler: Problem1/Variation1 – Add all the natural numbers below one thousand that are multiples of 3 or 5

Problem 1 on Project Euler was posted on October 5th, 2001 (just days before my birthday). I’m solving the problem nearly 10 years later! And since it’s one of the early ones, it’s quite simple. The problem description is as follows:

If we list all the natural numbers below 10 that are multiples of 3 or 5, we get 3, 5, 6 and 9. The sum of these multiples is 23.

Find the sum of all the multiples of 3 or 5 below 1000.

In honor of code katas and for the sake of reinforcing knowledge I already have, I will do several variations of solutions for this problem. I will try to intentionally choose the most unoptimized solution to start with (often a for loop) and work towards a solution that can scale to 1 million numbers and still be executed quickly. This will remind me of why certain solutions are poor, and give me an opportunity to slowly ramp into a good solution by exposing myself to different parts of the problem that surface when you evaluate the previous solution.

For the first variation, I chose to use a loop to cycle through all 999 numbers. The reason we’re only using 999 numbers is that the problem specifically states that it wants all natural numbers below 1000, which excludes 1000 itself. I am then applying modulus of 3 and 5 independently to the current iteration value and looking for a remainder. A result of 0 tells me that the number is evenly divisible by the number it is checked against (3 or 5). In these cases, I’m storing the current iteration value to an array for summation later on. If the current iteration value is evenly divisible by 3 and 5, we don’t want to add it to the array twice (which will throw off our summation) so we’re simply or’ing the modulus results.

Here’s the solution:

#!/usr/bin/ruby
#
# Project Euler
# Problem #1 - Add all the natural numbers below one thousand that are multiples of 3 or 5.
# Solution #1 - for loop
#
# Copyright 2011, Kevin Elliott, kevin@phunc.com
#

# Definitions
floor = 1
ceiling = 1000
multiples_of = [3, 5]

# Initializations
results = []

# Calculate
puts "Numbers below "+ceiling.to_s+" that are multiples of 3 or 5:"
(floor..ceiling-1).each do |i|
  if ((i % 3) == 0) || ((i % 5) == 0)
    results &lt;&lt; i
    puts "  "+i.to_s
  end
end

# Sum
sum = 0
results.each do |r|
  sum += r
end
puts "Sum = " + sum.to_s

Keep in mind that I have intentionally made it readable, because readable code is more useful to other people than super obfuscated and optimized code. I prefer to have code document itself, but for the sake of showing others I have put in some commenting. There are glaring ways to improve this, which I will begin to do in Problem1/Solution2.

What does your code look like?

March 16th, 2011

Code Katas and Project Euler

After reflecting on yesterday’s post on the importance of practice for a little while, I realized that I wanted to start doing some katas. A friend of mine mentioned Project Euler, which I had never seen before. Apparently over 150,000 registered users exist on Project Euler, which tells me there are a LOT of coders and students trying to practice their skills. Pretty fascinating really.

I am going to begin posting my attempts at solving various code katas as well as Project Euler submissions on this blog. It may be useless to some people, but I hope it inspires others to try to do katas every day. If you end up playing along, please post yours in the comments of the associated post!

March 15th, 2011

The Importance of Practice

Software developers who wish to excel should be craftsmen. This idea of craftsmanship in software development is not entirely a new concept, but it appears to be making a run around the software blogs. In a succinct but likely overly simplified description, craftsmanship is simply taking pride in your work and striving to be the best at what you do. This is opposed to simply being lazy at whatever you do and doing only what is necessary to get by. Being a craftsman means you are aiming to become the best at your trade, regardless of your circumstances. Most people will find it hard to admit that they’re lazy. People who really want to do well will immediately recognize and admit that they’re lazy, possibly approaching things incorrectly today, and look to find ways to better their mental approach to learning and implementation. Craftsmanship provides a framework for this by establishing some very simple but essential rules up front.

But some may still ask, why craftsmanship? What does this have to do with blacksmithing, leather making, or building houses? Everything. Software engineering has been viewed as an independent trade unlike other crafts for way too long. It’s time people begin to realize that all the things necessary to be the best in crafts like acting, design, and sports are required to be in the top category of software. Some of these very necessary responsibilities of being a craftsman are:

  • embracing and understanding your faults, failures, and inadequacies
  • striving to do better, not simply staying at status quo
  • practicing, practicing, practicing

Let’s look at each a bit closer. Some of this may seem very obvious to you, but the importance of each should not be reduced.

Admittance

Absolutely key is taking a leap and admitting to yourself that you’re not as good as you think you are. For those that are above the mediocre line already, this is a difficult task to perform regularly because these people are often told they are doing great, either by performance reviews or compensation. They’re not necessarily bad at what they do, but they’re more likely to tell themselves that they need little improvement. It’s a psychological trick.

For the craftsman, he’s willing to open up his mind to the possibility that there are countless people better at the trade than himself, and to overcome this he will examine his faults and come to terms with the need for constant improvement. Like a student of zen philosophies, it’s a lifelong process that never ends at a plateau of graduation. This is contrary to our school experiences that teach us that we reach plateaus of knowledge rewarded by diplomas and certificates, signaling that our learning process can essentially stop or slow down. A craftsman rejects this notion because he realizes that he’s never truly a master of the trade.

Sometimes this is disheartening because the idea that we’ll never reach the top of the mountain can be overwhelming. Maybe this is why people often fail at reaching the top of their game, sometimes just feet away from being truly successful. Getting over this mental hurdle is important. You need to simply figure out where you can use improvement, and make commitments with yourself that you’re going to change in some way. It’s not unlike a person who is overweight and has to finally come to terms that he’s overweight and all previous diets have failed. If he wants to lose the weight, he’s got to admit to himself that his approach has been wrong, and make a firm commitment to resolve it.

Doing Better

After a lengthy process of coming to grips with your true skill, and only then, can you really set a path for how to do better. You really just need to figure out what weaknesses you have, and find ways to improve on them. For some it means creating a daily learning time for 60 minutes a day to study areas they need work on. For others it means a significantly higher commitment of time and energy.

But the improvement process does not have to be dry and boring. It just means that you have to consciously aim at improving the areas you’ve identified as focus areas.

It may sound obvious to do better, but it takes a willful person to simply do it.

Practice

It’s true that by actually working on software development projects you are practicing. But you’re confined by walls of rules and foreign needs, which may actually block the ability to flex the creative juices you need to get value out of your practice time. Some people call commercial projects (that they work on at their jobs) prisons. They’re not entirely blasphemous for saying so. They’re legitimately experiencing a stifling of the ability to work on enjoyable practice opportunities. This comes to a detriment to them, because it leads down a path that prevents skill growth and the ability to think out of the box. Some employers don’t care about this because they’re not actually vested in the future talents of their employees… after all, employees who don’t perform well can simply be replaced. Most employers simply care about getting today’s work done, not next year’s. So if you’re slowly dried up of creative solutions, from the company’s perspective it’s really only going to hurt you when you get laid off or fired.

How depressing! What’s the solution? Practice! Not necessarily ‘on the job’ practicing, but practicing on your own. If you’re serious about being a craftsman, you’re going to admit that you need creative time to be able to improve. You also need to reinforce the knowledge you already have by practicing that as well. Professional football players are paid millions of dollars a year because they are one of just a few thousand in the world who are able to play at that level, and they get there by practicing every single day. They’re not just great athletes, they’re craftsmen.

Some great ways to practice software development are to perform daily rituals that allow you to practice, such as code katas, refactoring old code, and getting peer reviews to your solutions. Code katas allow you to tackle the same problems over and over, but in different ways each time. It gives you an opportunity to focus on small little fun projects without worrying about causing harm to some large commercial project. They are micro projects that help you to reinforce what practices you do are good, and which ones need improvement. It’s a remarkable way to learn how to do better, while still offering a creative outlet doing work that is often viewed as enjoyable.

Whichever way you prefer to practice, just do it. There’s no excuse. It only takes a few minutes a day to make a significant improvement to your skill-set and pave the road to craftsmanship.

More About Code Katas

November 10th, 2010

How to Develop Apps that Support iOS 3.x and 4.x on Both iPhone/iPod and iPad Platforms

Excuse the exceptionally long title of the article, but I just couldn’t figure out how to communicate this very specific example in any other way. In most cases, you want to develop your application once, but have it supported on all of the devices and firmware versions that Apple still supports. You don’t want to alienate your older user-base, but you still want to take advantage of all those great new API features to lure in new customers.

Apple could have been clearer about this because a lot of developers still seem to get into a panic about whether or not they can still support older devices when each new firmware version comes out. Yes, you can write applications that include iOS 4.2 features, but still runs on iOS 4.1 and 3.0-3.2 (albeit without the specific features of the newer versions).

To do this, you will just need to setup your Xcode project to support this.

To setup your application to support versions between 3.x all the way up to 4.x:

  1. Install the latest SDK. Unless you are avoiding beta releases, you should always have the latest GM SDK installed as your primary install. GM generally means Gold Master, which is a term from the CD/DVD days which means the official public release. You can always install beta SDKs into your home directory (I install mine in ~/SDKs), leaving /Developer for the latest GM SDK. Even if you are primarily building for older firmware versions, you should always been on the latest GM SDK because it offers fixes to Xcode and the libraries that you’re building against. Also, you will be able to take advantage of the new API features at any time should you want or need to.
  2. Set your Base SDK to the highest available version. It may seem like you will not be able to support 3.1.2 if you set this to 4.0, but this simply defines which set of SDK libraries and headers to use, of which will have support for many versions before it. You can also think of this as the “max supported version,” although the binaries will still run on newer firmware versions but you may have problems.
  3. Set your iPhone OS Deployment Target to the lowest version you wish the application to run on. Here’s where you get to decide how far back you’d like your application to run on. Beware though, just because you set it to 3.1.2 it doesn’t mean you can safely allow 4.0 APIs to be called. You will have to weak link UIKit, and then
  4. Check whether or not the current device supports features as you need them. In your code you will need to conditionally check against what features are available for the current device. For example, if you wish to call multi-tasking functionality, you will need to first check for it, and then provide code for devices that support multi-tasking and then code for devices that don’t. Since 3.x and 4.x are significantly different, this can get tedious but it’s what is required if you wish to support both major firmware version in a single codebase. You want to follow Apple’s guidelines and check for feature availability, and not simply check for firmware version. If you only check the firmware version, you could possibly blackhole some versions that might actually have the feature on it. This technique is smart because then you don’t need to know which firmware versions offer which feature, and you’re just checking to see if the feature itself exists.
  5. Check whether or not your current device supports functions. Similar to #4, you will want to first check to see if newer function and method calls are available before you actually use them. These checks will only be necessary for as long as you are supporting older major versions (i.e. 3.x) in your applications. Again, it seems tedious, but it’s very necessary.
  6. Test, test, test! I can’t stress this enough. Calling APIs that were introduced in 4.2 but are not available in 3.1.2 while the application is running on a device with 3.1.2 will result in immediate crashes. Thus, if you forget to wrap your 4.x functionality in a feature or function condition test, your users will experience crashes! Therefore it is incredible important that you test across multiple devices running multiple firmware versions. If you have a Base SDK of 4.2 and a Deployment Target of 3.1.2, then it will be a good idea to test on 3.1.2, 4.0, 4.1, and 4.2. Don’t skip this!

That’s it. Do you have any other tips to make this process easier? Shout out in the comments.

November 6th, 2010

A Must Have Deploy Script for Your Cocoa Mac Apps

Matt Gallagher over at Cocoa with Love just wrote:

Deployment for a Cocoa Mac application normally involves a few common steps: committing code into a repository, updating version numbers and packaging the application as a DMG disk image. In this post, I’ll show you a combination bash/perl/Applescript to handle all these tasks in a single script.

This recent article presents a script that every Mac developer should be using (or at minimum something similar to this) when they’re building their Mac applications. I can’t understate how great this is. It is simple in execution and seemingly unimportant, but it’s very valuable to your development and deployment workflow. By using this script he wrote, you will:

  1. Ensure you’ve committed your latest changes (and will exit if you haven’t).
  2. Increment your own version build number every time you build.
  3. Clean your application’s Release target, so that there is no weird cruft left behind, which will eliminate any weird bugs you may have occasionally.
  4. Build your application using the Release target.
  5. Tag this commit as the version (assuming you use git).
  6. Bundle up your app into a DMG file and will use the appropriate background.png file for the folder image (to show your users how to install; only if this file exists). It’ll clean up after itself and remove temp files it created.

If you’ve already been building Mac applications, or you’re about to start building for the Mac App Store, you owe it to yourself to set this up. You can download the bash script at his article. Thank Matt (@cocoawithlove) once you’ve used it for a little while and see the value of it, as I’m sure he’ll appreciate it.

November 3rd, 2010

As a Developer: Why I Still Prefer the App Store over the Android Market

If you’re a developer, you have no doubt spent time thinking about whether you will publish mobile apps on the Apple App Store or the Google Android Market, and where you will spend a majority of your time. As an indy developer, you know that it’s important to maximize your profits in order to stay alive. For most indy devs, it’s really not possible to develop for both platforms, at least in the beginning. They use different languages, frameworks, and have a differing culture to boot. Business models are slightly different, and you need to follow different guidelines from each of the companies.

Having a long running UNIX background, I have a lot of respect for the open source community and the idea of openness has tremendous value to me. However, I also have a family to feed and a roof to put over my head, and just have not seen enough proof that Android Market has just as much potential as the Apple App Store. It is vital that in order to continue product innovation I need to make a meaningful profit. Taking the risk to develop on the Android Market is quite high considering the countless articles have displayed losses and dismal profits on it. But still, I’m shamed by users for not building apps on the Android platform.

I have made an effort to have conversations with a variety of people on both sides of this issue, and of friends and foes. Since I’m for the App Store (for the time being), I tend to agree more with App Store proponents and disagree with Android Market proponents. Other developers understand my woes, especially if they’ve already taken great risks to build their own companies instead of simply working for someone else’s development shop. As the owner of a company, it’s your duty to determine the best path of revenue generation for your company and to maximize your profitability, while still reaching the goals you’ve set. So it’s really important that you actually take a look at how both stores are actually doing, regardless of the “handset counts” that both Apple and Google like to sling at us. I find that when I defend my position to end-users and friends that have Android phones, I come back to these unrefutable points over and over again:

  1. Google has lured in Android users by heavily marketing that it’s an open platform. People who prefer open platforms (like Linux for example) tend to dislike paying for things. This is why third-party commercial add-on software for Linux has failed outside of the Enterprise market time and time again. Thus, only free or hyper-cheap apps do well on the Android Market. I’m not saying that all users of an open platform are “cheap,” but a generalization can be made that they typically are.
  2. Google continues to copy Apple and does not do enough innovation of their own. Everything about the Android platform feels like a cheapened iOS experience. The popup keys on the keyboard, the sliding table views, and even the basic lock screen concept. It’s great that they took “the best of Apple’s creation” but they haven’t done a remarkable job of innovating past it. There are those that disagree and say that Google has made some incredible innovations on Android, but frankly they’ve yet to prove it to me.
  3. Apple manages the entire process of product design, development, implementation, testing, manufacturing, management, customer support, and some of the retail experience. They even make their own chips occasionally! They’re able to finely tune every aspect of this process that they create excellent products every time. And when they screw up just a little bit, there’s such a high expectation from their customers that there is a backlash, and Apple generally is held responsible for making some kind of solution. You can not say this about Google and it’s manufacturing partners. Not only must Google anticipate what its manufacturing partners will want to bring to the table hardware-wise, it has to support relationships with a large base of manufacturers. Akin to Microsoft, this eventually breaks down (look at the driver mess that Windows eventually grew to that later took many years to simplify; it’s still a problem to this day). This is because hardware manufacturers are not selling Android, they’re selling Android hardware, so they need to distinguish themselves amongst the wide variety of other Android hardware. They’ll eventually become more and more aggressive, putting hardware in that other manufacturers will be unable or unwilling to do, and thus you’ll see what happens in #4.
  4. Since Google only focuses on the architecture and software platform for Android, it partners with hardware manufacturers to create handsets (and now tablets). This leads to hardware fragmentation, where some devices have feature X and other devices have feature Y. There is nothing preventing these manufacturers from putting in hardware so much more effectively (through quantity deals with suppliers) than other manufacturers that you’ll begin to see more and more cases of features only available on some devices. This will lead to developers only focusing on core commonalities, because being able to support all the features will simply be out of budget and out of scope. Should Google mandate hardware specifications for every single device that manufacturers create, then this may become less of a problem, but I don’t believe it would allow manufacturers to distinguish themselves appropriate if Google held that responsibility tightly.
  5. Android’s software development kit (SDK) allows for much lower-level access to the operating system than iOS does. That’s a mixed bag for sure. While the reduced restrictions allow for more interesting applications that integrate into the operating system (such as a replacement loader screen, or a themed lock screen), it also leads to a reduced user experience, with lots of applications crashing or slowing down the phone. Applications are not jailed into safe areas on Android as they are on iOS. As a developer, Android is quite interesting and alluring because of it’s open access, but end-users will interpret problems with your app typically as caused by your app and not of another app, even when it’s some other intrusive app causing the performance issue. As a developer, I want to use Android from a technical perspective because of it’s nearly unrestrictive layer of access, but as a businessman I know that I won’t make as much money when users blame my app for causing a problem that another app did. This is so much less of a problem on iOS.
  6. Lack of hardware specific optimizations leads to a waffled experience across the varying Android hardware selection. For a while, Android wasn’t even optimized at all (*cough* 1.5, 1.6 *cough*) and hardware manufacturers put beefy hardware in that was even more compelling than iPhone/iPod hardware. Users bought purely on those specifications, and thus they were disappointed when they sat down with their iPhone friends because the experience just wasn’t the same. Sadly, many of those users never even got an upgrade to the more optimized version of Android (2.1) because their carrier and/or hardware manufacturer chose not to do so. Forced obsolescence.
  7. More market share does not equate to a better product and experience. A lot of users are buying Android products because their cellular carrier doesn’t have iPhones. Others are lured in by its openness or lower price. Still others buy Android because they see exciting hardware specifications. These reasons do not inherently point to a superior product, just an available one. Just because more users existing on a platform does not necessarily make it a better cause to focus on it. Some people will not see this, as I have noticed in my conversations with Android fanatics, but I don’t expect them to put their biases down either. To some people, popularity is a sure sign that a product is superior, and there’s nothing you can really do to sway those people. They’re basing their decisions on that. Others prefer a solid product, with a clear vision and a proven track record for innovation, and are willing to pay for it. That diversity is great, and will keep the market fluid.
  8. Apple iOS has a superior user interface and user experience for the user and for the developer. Apple has a great set of policies about their design and architecture from the top, and they insist that it permeates to the bottom. Apple spends a considerable amount of time focusing on how the user will experience the platform, and helps to guide and enforce developers to continue that experience in their applications. While pundits claim that Apple is too restrictive, you can see that it can be fairly lenient on content by simply looking at the cruft that does make it through. They’ve made everything really easy to use, without sacrificing the power of the platform. This even shows in their sample code and their software frameworks. The API is mostly brilliant. On the other hand, Google has made a platform by geeks for geeks. They haven’t spent enough time thinking about the millions and millions of untechnical users, who will continue to remain confused by the entire platform. They’ve missed out on opportunities to lure their less technical customers into purchasing apps (which makes us, the developers, happy) because their customers struggle to understand what apps are, how to get to them, how to configure them, and whether or not they can install it on their internal or external memory cards. Some developers don’t want to deal with “dumb customers,” so hey, that’s fine by me. More for us on the iOS platform, where those not as familiar with technology can feel comfortable about trying it out. But it’s a shame that Google doesn’t hire two really important people: a solid UI/UX design genius and a marketing architect who can simultaneously push the product while also educate the customers on how to use it.

I do hope that Google matures the Android platform and the Android Market. When it does become ubiquitous, but more importantly profitable, I will absolutely begin to consider porting my products to the platform. I want to develop for Android, as I’m certain that it will soothe my inner geek. I’m interested in continuing the discussion of my opinions in this article moving forward, and I encourage everyone to engage in commentary. As these are only my opinions and views, I don’t insist that I am right in every way and I’m open to considering other viewpoints. After all, I just want to make excellent apps that people will use and make a decent living at the same time. An open dialogue should help to make that happen.

September 27th, 2010

Migrating a Rails app from MySQL/PostgreSQL to MongoDB

NoSQL databases are becoming immensely popular. They’re the in thing right now. That alone isn’t a good reason to choose to use a piece of technology, but there are a lot of other advantages to using NoSQL in attribute rich data applications. There are already a ton of resources discussing why you should choose NoSQL over an RDBMS and which one you might even want to use, but I’m not going to rehash that here. I’ll assume you’ve made the choice to specifically migrate from MySQL or PostgreSQL to MongoDB.

1. Prerequisites

  • Rails 2.3.9 or 3.0.0 or higher
  • MySQL or PostgreSQL plus a supporting gem
  • MongoDB
  • MongoMapper gem

2. Choose your migration strategy

Everyone’s project needs are different, so you are going to have to spend some time assessing just how you will be using MongoDB, and whether or not you will even completely migrate over to it. There are a lot of reasons to keep data in an RDBMS still, just like there are a lot of reasons to continue using HAML files, filesystem caches, and memcached. Chances are, NoSQL might really only be beneficial to you for a portion of your data. I can’t decide this for you — you will need to decide this for yourself.

  • Supplemental MongoDB. You want some new data types stored to MongoDB, but do not intend on migrating existing data over and will continue to service it from RDBMS. This new data is not strictly tied to your existing data sets, and will likely not need extensive connections to it in the future.
  • Archival MongoDB. You only want to archive/mirror data in your RDBMS to MongoDB for future reporting purposes, or to serve as a live backup mechanism. This allows you to copy data into MongoDB and then annotate it since there is no schema.
  • Partial Migration. You want to move some of the data from your RDBMS into MongoDB, and leave some data back in the RDBMS. (This is similar to Supplemental, but I see some differences, namely how the data in both databases are related to each other)
  • Full Migration. You want to completely migrate to MongoDB, and eradicate your RDBMS from the solution altogether.

For a recent project I have been working on, I determined that the NoSQL document hierarchy and lack of strict schema of MongoDB was perfect for the primary data that is already being stored. I wanted to perform a Full Migration to MongoDB, but only after I had a chance to learn and evaluate MongoDB in both development and production environments. To accomplish this, I first performed a Partial Migration, with intention to later go to a Full Migration.

One of my goals was to be able to fall back to my RDBMS-only solution should the experiment using MongoDB be unsuccessful. To be able to accomodate this, I had to:

  1. create a new git branch called ‘mongo’ to fork the code giving me a safe place to experiment
  2. keep my Rails classes tied to PostgreSQL in app/models
  3. create a library in a new namespace, specifically for the MongoMapper classes
  4. create a single migration that performed all of the migrations at once (although, only one-way into MongoDB)

3. Create your MongoMapper classes

Creating classes for MongoMapper require a little more work than traditional ActiveRecord ones. You’ll be happy to know that most of the concepts involving relationships carry over, and only require minor changes in order for them to work. While a schema is not required, in order to access the attributes via accessors, you’re going to need to define keys (equivalent to regular attributes). The reason for this is that since there is no schema, MongoMapper can not automatically determine which attributes exist, since it can vary per instance of document (“database row”).

Another area that is tricky to many is the concept of Document versus EmbeddedDocument. An EmbeddedDocument is only stored inside of a Document in MongoDB. Thus, you can not directly query for instances of an EmbeddedDocument without first examining it’s parent Document.

4. Build your migration

If you’re simply adding new data, you do not need to migrate any old data. However, if you want to pull data from your old RDBMS into Mongo, you’re going to need to write a migration that can retrieve data from the RDBMS as well as store data into MongoDB via your custom MongoMapper classes.

  • create a migration
  • load data from the RDBMS using the ActiveRecord models
  • create objects using the MongoMapper models
  • associate the objects as necessary to preserve the relationships of the data (2 way required in MongoDB!)

Because the structure of this migration can vary so significantly between projects, I have not included a particular structure or example. I may share an example in a future article that shows how this data is migrated.

What strategy do you use to migrate from an RDBMS to MongoDB?