Persisting a Todo List With MongoDB and Geddy

In this three part tutorial, we’ll be diving deep into creating a to do list management app in Node.js and Geddy. This is the last entry in the series, where we’ll be persisting our todo items to MongoDB.

As a quick refresher, last time, we created our todo resource and made a working to do list application, but the data only existed in memory. In this tutorial, we’ll fix that!

Intro to MongoDB

MongoDB is a NoSQL document store database created by the folks over at 10gen. It’s a great database for Node apps because it stores its data in a JSON-like format already, and its queries are written in JavaScript. We’ll be using it for our app, so let’s get it set up.

Installing MongoDB

Go to http://www.mongodb.org/downloads and download the latest version for your OS. Follow the instructions in the readme from there. Make sure that you can start mongod (and go ahead and leave it running for the duration of this tutorial)

It’s worth noting that you’ll need to have mongo running any time you want your app running. Most people set this up to start up with their server using an upstart script or something like it.

Done? alright, let’s move on.

MongoDB-Wrapper

For our app, we’ll be using a module that wraps the mongodb-native database driver. This greatly simplifies the code that we’ll be producing, so let’s get it installed. cd into your app and run this command:

npm install mongodb-wrapper

If all goes well you should have a mongodb-wrapper directory in your node_modules directory now.

Setting up Your Database

Mongo is a really easy DB to work with; you don’t have to worry about setting up tables, columns, or databases. Simply by connecting to a database, you create one! And just by adding to a collection, you make one. So let’s set this up for our app.

Editing your init.js file

We’re going to need access to our DB app-wide, so let’s setup our code in config/init.js. Open it up; it should look like this:

// Add uncaught-exception handler in prod-like environments
if (geddy.config.environment != 'development') {
  process.addListener('uncaughtException', function (err) {
    geddy.log.error(JSON.stringify(err));
  });
}
geddy.todos = [];
geddy.model.adapter = {};
geddy.model.adapter.Todo = require(process.cwd() + '/lib/model_adapters/todo').Todo;

Let’s add in our db code at the very top (and remove the geddy.todos array while we’re at it):

var mongo = require('mongodb-wrapper');

geddy.db = mongo.db('localhost', 27017, 'todo');
geddy.db.collection('todos');

// Add uncaught-exception handler in prod-like environments
if (geddy.config.environment != 'development') {
  process.addListener('uncaughtException', function (err) {
    geddy.log.error(JSON.stringify(err));
  });
}
geddy.model.adapter = {};
geddy.model.adapter.Todo = require(process.cwd() + '/lib/model_adapters/todo').Todo;

First, we require the mongodb-wrapper module. Then, we set up our database, and add a collection to it. Hardly any set up at all.

Rewriting Your Model-Adapter

Geddy doesn’t really care what data backend you use, as long as you’ve got a model-adapter written for it. This means that the only code that you’ll have to change in your app to get your todos into a database is in the model-adapter. That said, this will be a complete rewrite of the adapter, so if you want to keep your old in-memory app around, you’ll want to copy the code to another directory.

Editing Your Save Method

Open up your model-adapter (lib/model_adapters/todo.js) and find the save method. It should look something like this:

this.save = function (todo, opts, callback) {
  if (typeof callback != 'function') {
    callback = function(){};
  }
  var todoErrors = null;
  for (var i in geddy.todos) {
    // if it's already there, save it
    if (geddy.todos[i].id == todo.id) {
      geddy.todos[i] = todo;
      todoErrors = geddy.model.Todo.create(todo).errors;
      return callback(todoErrors, todo);
    }
  }
  todo.saved = true;
  geddy.todos.push(todo);
  return callback(null, todo);
}

Make it look like this:

this.save = function (todo, opts, callback) {
  // sometimes we won't need to pass a callback
  if (typeof callback != 'function') {
    callback = function(){};
  }
  // Mongo doesn't like it when you send functions to it
  // so let's make sure we're only using the properties
  cleanTodo = {
    id: todo.id
  , saved: todo.saved
  , title: todo.title
  , status: todo.status
  };
  // Double check to see if this thing is valid
  todo = geddy.model.Todo.create(cleanTodo);
  if (!todo.isValid()) {
    return callback(todo.errors, null);
  }
  // Check to see if we have this to do item already
  geddy.db.todos.findOne({id: todo.id}, function(err, doc){
    if (err) {
      return callback(err, null);
    }
    // if we already have the to do item, update it with the new values
    if (doc) {
      geddy.db.todos.update({id: todo.id}, cleanTodo, function(err, docs){
        return callback(todo.errors, todo);
      });
    }
    // if we don't already have the to do item, save a new one
    else {
      todo.saved = true;
      geddy.db.todos.save(todo, function(err, docs){
        return callback(err, docs);
      });
    }
  });
}

Don’t be too daunted by this one; we started with the most complex one first. Remember that our save method has to account for both new todos and updating old todos. So let’s walk through this code step by step.

We use the same callback code as we did before – if we don’t have a callback passed to us, just use an empty function.

Then we sanitize our todo item. We have to do this because our todo object has JavaScript methods on it (like save), and Mongo doesn’t like it when you pass it objects with methods on them. So we just create a new object with just the properties that we care about on it.

Then, we check to see if the todo is valid. If it’s not, we call the callback with the validation errors. If it is, we continue on.

In case we already have this todo item in the db, we check the db to see if a todo exists. This is where we start to use the mongodb-wrapper module. It gives us a clean API to work with our db. Here we’re using the db.todos.findOne() method to find a single document that statisfies our query. Our query is a simple js object – we’re looking for a document whose id is the same as our todos id. If we find one and there isn’t an error, we use the db.todos.update() method to update the document with the new data. If we don’t find one, we use the db.todos.save() method to save a new document with the todo item’s data.

In all cases, we call a callback when we’re done, with any errors that we got and the docs that the db returned to us being passed to it.

Editing the all method

Take a look at the all method, it should look like this:

this.all = function (callback) {
  callback(null, geddy.todos);
}

Let’s make it look like this:

this.all = function (callback) {
  var todos = [];
  geddy.db.todos.find().sort({status: -1, title: 1}).toArray(function(err, docs){
    // if there's an error, return early
    if (err) {
      return callback(err, null);
    }
    // iterate through the docs and create models out of them
    for (var i in docs) {
      todos.push( geddy.model.Todo.create(docs[i]) )
    }
    return callback(null, todos);
  });
}

Much simpler than the save method, don’t you think? We use the db.todos.find() method to get all the items in the todos collection. We’re using monogdb-wrapper’s api to sort the results by status (in decending alphabetical order) and by title (in ascending alphabetical order). Then we send that to an array, which triggers the query to start. Once we get our data back, we check to see if there are any errors, if there are, we call the callback with the error. If there aren’t any errors we continue on.

Then, we loop through all the docs (the documents that mongo gave back to us), create new todo model instances for each of them, and push them to a todos array. When we’re done there, we call the callback, passing in the todos.

Editing the load method

Take a look at the ‘load’ method, it should look something like this:

 this.load = function (id, callback) {
  for (var i in geddy.todos) {
    if (geddy.todos[i].id == id) {
      return callback(null, geddy.todos[i]);
    }
  }
  callback({message: "To Do not found"}, null);
};

Let’s make it look like this:

this.load = function (id, callback) {
  var todo;
  // find a todo in the db
  geddy.db.todos.findOne({id: id}, function(err, doc){
    // if there's an error, return early
    if (err) {
      return callback(err, null);
    }
    // if there's a doc, create a model out of it
    if (doc) {
      todo = geddy.model.Todo.create(doc);
    }
    return callback(null, todo);
  });
};

This one is even simpler. We use the db.todos.findOne() method again. This time, that’s all we have to use though. If we have an error, we call the callback with it, if not, we continue on (seeing a pattern here yet?). If we have a doc, we create a new instance of the todo model and call the callback with it. That’s it for that one.

Editing the remove method

Take a look at the remove method now, it should look like this:

this.remove = function(id, callback) {
  if (typeof callback != 'function') {
    callback = function(){};
  }
  for (var i in geddy.todos) {
    if (geddy.todos[i].id == id) {
      geddy.todos.splice(i, 1);
      return callback(null);
    }
  }
  return callback({message: "To Do not found"});
};

Let’s make it look like this:

this.remove = function(id, callback) {
  if (typeof callback != 'function') {
    callback = function(){};
  }
  geddy.db.todos.remove({id: id}, function(err, res){
    callback(err);
  });
}

The remove method is even shorter than it used to be. We use the db.todos.remove() method to remove any documents with the passed in id and call the callback with an error (if any).

Time for the Magic

Let’s go test our app: cd into your project’s directory and start up the server with geddy. Create a new todo. Try editing it, have it fail some validations, and try removing it. It all works!

Conclusion

I hope you’ve enjoyed learning about Node.js, MongoDB and especially Geddy. I’m sure by now you’ve got a million ideas for what you could build with it, and I’d love to hear about them. As always, if you have any questions, leave a comment here or open up an issue on github.