Building a voice app with Node, Twilio, Compose and Heroku - Part 2

So in my previous post I gave you a brief overview of what it was that I was trying to achieve here. Use a load of decent APIs to create a service that doesn’t exist right now (probably for a reason?).

The idea is to create a service that I can call from a phone and retrieve phone numbers that I’ve put in to a website, and be connected. You enter in a user ID and your PIN code, and you can choose from a list, and eventually be put through. This could be useful if your phone has run out of battery or you can’t get to your contacts for some reason. As I said, this might not be useful to anyone - but my phone is always giving up on me so I thought might as well try.

Setting up the project #

So when express-generator creates your project - it gives you a load of scaffolding structure for your site - and for us, that’s perfect. The structure looks like this:

twilio-numbers/
-- bin/
-- node_modules/
-- public/
-- routes/
-- views/
-- app.js
-- package.json

Getting your hands dirty #

Great. So lets get started. I created a file called twilio.js in my routes folder, and in app.js I changed the lines on 8-9 to look like this:

var routes = require('./routes/index');
var twilio = require('./routes/twilio');

routes will hold the frontend routes, login, manage, logout etc, and twilio will hold the endpoints for Twilio to call to retrieve our script file.

Now, in twilio.js, I added the following

var express = require('express');
var router = express.Router();

// Twilio
var twilio = require('twilio');

// Connect and initialise database
var mongoose = require('mongoose'),
    Schema = mongoose.Schema;

// Connect to Compose
mongoose.connect('mongodb://<username>:<password>@<server>.0.mongolayer.com:10024,<server>.1.mongolayer.com:10024/<db-name>?replicaSet=set-540eca6a792b9d2e29004d94');

I’m including the Twilio and Mongoose models and aliasing the Mongoose Schema object to Schema - you’ll see why in a minute!

If you’re following along, I’d advise that you change your connection string to match the one that Compose gives you when you sign up. Now, lets get into the data structure - there’s a couple of things I’d like you to know before I carry on with the walkthrough.

Let’s talk schemas #

So, a schema is a way for a schema-less database platform like MongoDb to enforce some sort of structure.

In MySQL, for example, you need to design and initialise your tables first before you add any data to them. This is good because it means you have the opportunity to sit down and plan what the hell it is you’re going to chuck at this table. It ensures consistency and allows you to make some upfront decisions about what datatypes and validation you’re going to enforce.

Now with Mongo, things are a little bit more laid back. Mongo doesn’t really care what you put in a collection (this is Mongo terminology for a table, SQL people!) - you are pretty much free to put whatever you want in, whenever, even if documents (SQL: row), are meant to be the same, so you could do a find() and retrieve a bunch of randomly formatted rows that might not even share a common property (SQL: column).

This has its advantages, it means that while you will generally follow the same structure, you might embed a bunch of stuff into one document that might not exist in another, whereas in SQL you’d have to do a bunch of joins.

Example #

This is handy for a classifieds site, for example - where one row might represent a car, with properties like horsepower, engineLitres, doors and colour, for example. Another document in the same collection might be a computer, which will have properties like cpu, memory, hdd, etc. MongoDB allows you to create these documents without much complaint, which leads to some nice side effects.

A query might look like this:

classifiedItems.find({"engineLitres": "1.5"}).exec(console.log);

You could also search the same collection for computers:

classifiedItems.find({"cpu": "3ghz"}).exec(console.log);

This makes for some interesting options when its comes to searching and presentation. But I digress!

Why we don’t want to do that #

This leads to inconsistent documents. Consistency is key. Makes for more repeatable, understandable, testable code - and I’m a stickler for that.

Enter Mongoose Schemas #

Mongoose is a npm module that allows for easy interaction with MongoDB databases. It is a wrapper around the native MongoDB query language and provides us with some nice functions that make our lives easier. And we all want our lives to be easier. It also provides us with the schemas I was just banging on about..

Have a read of the Mongoose documentation, or read on:

Defining a schema #

Here’s a schema that I’m going to use for our app. It’s a user schema:

var userSchema = new Schema({
    userCode: Number,
    userPIN: Number,
    email: String,
    password: String
});

Lets step through what’s going on here:

We’re defining this in Mongoose Schema language (which is basically just a key/value object with our property names and expected data-types. These schemas are far more powerful than this, but for our first exploration into them, lets keep it simple.

I then register the schema with Mongoose and everything is great:

var User = mongoose.model('User', userSchema);

So, at the moment my code looks like this:

var express = require('express');
var router = express.Router();

// Twilio
var twilio = require('twilio');

// Connect and initialise database
var mongoose = require('mongoose'),
    Schema = mongoose.Schema;

// Connect to those fine folks @ Compose    
mongoose.connect('mongodb://<username>:<password>@<server>.0.mongolayer.com:10024,<server>.1.mongolayer.com:10024/<db-name>?replicaSet=<set>');

// Set up our basic user collection
var userSchema = new Schema({
    userCode: Number,
    userPIN: Number,
    userName: String,
    email: String,
    password: String
});
var User = mongoose.model('User', userSchema);

module.exports = router;

Great - next time we’re going to add some endpoint functionality to this and get talking to Twilio!