Saturday, November 26, 2011

Heroku + Node.js + MongoLab


I've taken a little break from the hardware side of things to mess with some software. Once I got started it was hard to stop. One of the things that I love about software is the instant feedback and ease of debugging. No connections to check or fleks of solder to worry about. Just write some code, compile when required, and run it. I never have to leave my seat which is great over the Thanksgiving break. Anyway, I wanted to post a quick how-to on setting up a mongo database on the Heroku platform. Heroku is where I'd like to run a server for WiFi enabled devices. Devices with internet access can send packets to the server that will be handled appropriately. I've really been trying to learn a ton about Node and MongoDB lately so those are the technologies I decided to mess around with.

Heroku

If you have never heard of or used Heroku then you haven't lived. It is a fully Git-integrated hosting solution. Like the heat and eat sausages at the supermarket, Heroku is pretty much commit and push and you are live on the internet. I can't say enough good things about how easy it is to deploy with Heroku and it can scale as fast as you can request additional resources from the command line. The other nice thing is that they offer a free tier for people to mess around with it. Anyway, enough marketing for Heroku. It's great and you should try it. They have some great tutorials on how to get going so I won't spend time on that here. See this page for a great walkthrough.

MongoLab

So it's no secret that there is this whole NoSQL movement going on and if you think it's just a fad then you need to take a closer look. A NoSQL-flavored database may be just what you need in certain scenarios. Sometimes the simplicity and easy setup of a document-based DB is just what you need. There are a lot of up and coming document-based DBs and I don't really have a great reason as to why I chose MongoDB other than I felt like it was easy to learn and has a lot of good documentation online. Since your Heroku app is completely contained within an Amazon EC2 instance you have to make use of various addons to get functionality. MongoLab is one of those addons that gives you access to a hosted mongo database. Again, the free tier is quite adequate for a new project.

Hooking It All Up

So all I wanted to do was get a Node.js app hosted that connected to a MongoDB database for the back end. What I ended up with was quite awesome and a breeze to use once set up. The steps I'm going to give are a little high level but if you have questions or want more detail just leave a comment or shoot me an email and I'll get back to you. I'll assume that you've set up your Heroku account and have the Heroku toolbelt installed at this point. You should also have installed Node and npm. There are great tutorials for doing those elsewhere.
  1. Create a new git repository
  2. $ git init
  3. Create a heroku app with the Cedar stack
  4. $ heroku create --stack cedar
  5. Add MongoLab AddOn
  6. $ heroku addons:add mongolab:starter
  7. Create server.js
  8. This is the meat of your Node.js application. I use the Connect library because it's simple to use and keeps the code clean. For the MongoDB interface I use the node-mongodb-native driver. The code for the server.js file can be found in a Gist I created.
  9. Create package.json
  10. This is how Heroku knows your app is a Node.js app.
    {
      "name": "mongolabtest",
      "version": "1.0.0",
      "description": "Sample app using Heroku, Node.js, and MongoLab",
      "engines":{
        "node": "0.6.x",
        "npm": "1.0.x"
      },
      "dependencies": {
        "connect": "1.8.x",
        "mongodb": "0.9.x"
      }
    }
  11. Create Procfile
  12. This is literally a file named "Procfile" in the root of your application directory. It is used by Heroku to know how to start your application in the EC2 instance. Our Procfile will tell Heroku to use a web dyno to run our node app. All that goes in this file as:
    web: node server.js
  13. Have NPM Install Dependencies
  14. NOTE: You may want to add "node_modules" to a .gitignore file to prevent all of the module code from being added to the git repo and being pushed to Heroku.
    $ npm install
  15. Commit Your Code
  16. Heroku is intimately tied to Git so to get your code to Heroku it has to be in the form of a git commit.
    $ git add .
    $ git commit -m "Initial Commit"
  17. Deploy It
  18. That's it. We're ready to send it to Heroku where they will work some special sauce goodness on it
    $ git push heroku master
  19. Marvel In Your Awesomeness
  20. Use the heroku open command to launch your app in a web browser. This simple app will parse a query string and add a document to the database with all of the arguments. It then returns the JSON of the object back to the browser.
    $ heroku open
  21. Check Out the MongoLab Admin Tools
  22. To verify your request generated an item in the database just log in to Heroku and access the MongoLab addon from your app area. You should see something like this:
    Clicking on the "requests" collection will show a list of the documents.

Conclusions

How much easier can it get? If you follow those instructions and use the code in the Gist it will take you less than 5 minutes. The example I generated isn't horribly useful but you can see that it's a decent starting point waiting for enhancement. If that interested you I recommend digging deeper into Node.js, MongoDB, Heroku, and all of the tooling surrounding these things. It really makes programming a lot of fun.

15 comments:

  1. Great simple tutorial. Really appreciate it.

    ReplyDelete
  2. Thanks for the comment. Glad it was useful.

    ReplyDelete
  3. I followed this tutorial and at the end when I tries to run it the app crashed. Apparently it thinks db in server.js (line 7) is null since I got a type Error saying "Cannot call method 'addListener' of null". Why is that?

    Thanks for your help.

    ReplyDelete
  4. Sounds like there is an issue when it tries to do mongo.connect. I would probably look at the value of error in the callback. It may shed some light as to what is going on. Also, make sure that process.env.MONGOLAB_URI resolves to your MongoLab database location. You can check that from the command line by doing "heroku config | grep MONGOLAB_URI" I'd be interested to know what those things turn up.

    ReplyDelete
  5. Thank you for your quick response. I am new to node, heroku and mongo, so I might get something wrong.

    I tried "heroku config | grep MONGOLAB_URI" and it showed me a long url-type thing, I guess that's my connection string.

    The error says "node.js:201 throw e; // process.nextTick error, or 'error' event on first tick TypeError: Cannot call method 'callListener' of null". I tried to look that up on Google but it did not help. Any ideas? Also, you said "This simple app will parse a query string and add a document to the database with all of the arguments. It then returns the JSON of the object back to the browser." I am wondering how that will work? I am sorry if I asked some silly questions. I am just new to everything here.

    ReplyDelete
  6. You're right, the long url string is your connection string so it sounds like you have mongolab setup correctly in the heroku environment. Do you have the files exactly like I show them in the Gist? Did you create the heroku app with the cedar stack (heroku create --stack cedar)? Also, something you could try would be to run it locally. What OS are you running? You could install mongo and nodejs locally and test the server locally so eliminate the possibility that something goes wrong in the heroku environment. As far as how the simple app works it will just take the querystring you pass it so for example http://something.heroku.com?name=Dude&color=green would result in a new object in the database of {name: "Dude", color: "green"} and it will return that JSON to the browser. Hopefully that makes sense. Let me know if that helps.

    ReplyDelete
  7. Yes I copied all the files in the Gist and I created the heroku app with the cedar stack. I guess it's the heroku environment since I had mongodb and nodejs working together locally earlier this month. I tried this guy's code (http://experiencecraftsmanship.wordpress.com/2012/01/06/heroku-node-js-mongodb-featuring-the-native-driver/) and it seemed to be working (I didn't do the curl part since I don't have it installed, but when I do "heroku ps" it says the server is up). But he used Mongoose, which I am not planning on using.

    I am running Windows 7 32-bit if it helps.

    Also, one more silly question, is that "http://something.heroku.com" in the example you gave my app's url?

    ReplyDelete
  8. The url I gave (something.heroku.com) was just an example. You would want to use your heroku app url. After pushing your app to heroku and having it fail could you post the results of running "heroku logs".

    ReplyDelete
  9. Thank you for your help. I am not allowed to post more than 4,096 characters in the comment area, so I sent you an email with the logs. I hope it does not bother you. Again, thanks for helping me.

    ReplyDelete
  10. In researching K's issue it turns out that the latest version of the mongodb package does not work as before and the process will get hung up in the mongo.connect function. A work around for this is to fix the version in package.json to be 0.9.7. When I have time I will research why the newest package version no longer works.

    ReplyDelete
  11. After some follow up the reason the newest version of mongodb doesn't work is because it's apparently unsupported in the cedar stack environment. When trying to push the app the error is right there in the push. "npm ERR! error installing mongodb@0.9.9-6 Error: Unsupported" To work around this we need to add an "engines" section to our package.json to specify which versions of node and npm to use. After that everything works fine.

    ReplyDelete
  12. Updated post and Gist to include "engines" section to fix mongodb version issue.

    ReplyDelete
  13. Hi,

    Can I run monngodb commands ( or a javascript ) directly on heroku ??
    basically I don't want to start node just for processing my data ( some scheduled changes).

    thank you.

    ReplyDelete
    Replies
    1. I'm not sure I understand what you're trying to accomplish. If you're using MongoLab you can connect to that directly outside of the heroku environment and make changes. Heroku does have a command line you can attach to but it's more for running heroku commands. If you gave me a little more info I might be able to answer the question a little better.

      Delete
  14. I had to use the 1.9.x version of connect

    ReplyDelete

Keep it clean and civil. That's all I ask.