Introducing Monogram, the Anti-ODM for Node.js and MongoDB

In 2015, I started working on a new ODM for MongoDB and Node.js. It was based on the now-obsolete Object.observe() function, and I unfortunately had to scrap the whole project when the Object.observe() proposal was unexpectedly withdrawn. A lot of the core logic from the original ODM lives on in Archetype. But my brief time in Redux land and studying falcor taught me a crucial lesson: the Model-View-Controller paradigm is not the only way to do things. MVC and ODMs still make sense, but we can build a stronger, more functional abstraction as the basis for a more concise architecture. The new monogram has now been powering 100% of Booster's API queries for weeks and it's officially production ready. Here's what monogram is all about.

Function Calls are Objects

The mind-bending idea that made ORMs so popular in the late 90's and early 2000's was that you could merge object-oriented programming with RDBMS by mapping objects in your programming language of choice (typically Java for early ORMs) to rows in your database. In other words, an ORM let you think of database rows as objects, and the MongoDB equivalent ODM lets you think of MongoDB documents as objects. Unfortunately, even with Object.observe() it's tricky to get this mapping right in the general case because of in-place updates, not to mention the difficulties of distributed programming.

What if, instead of mapping documents to objects, we mapped database operations to objects? What if, instead of scratching our heads about how to make arrays and dates a nail for the OOP hammer, we took a page out of aspect-oriented programming's book and thought about how to better instrument our database operations? The key idea of monogram is that every function call on a MongoDB collection can be represented as an object that we call an action.

const { connect } = require('monogram');

test().catch(error => console.error(error.stack));

async function test() {
  const db = await connect('mongodb://localhost:27017/test');

  const Test = db.collection('Test');

  Test.pre(action => {
    /* Here's what an action looks like:
     * { _id: 59722626c9d79b6d1cbe0387,
     *   params: [ { hello: 'world' } ],
     *   collection: 'Test',
     *   name: 'insertOne',
     *   chained: [] }
     */
    console.log(action);
  });

  await Test.insertOne({ hello: 'world' });
}

Monogram defines actions for every operation in the MongoDB CRUD spec. This certainly makes logging easy, the above code is all you need to log every single database operation to the console. When you attach middleware to your collection like above, your middleware executes before the actual database function executes. In other words, the console.log() above happens before the insertOne() gets sent out to MongoDB. What about if you wanted to print out how long each operation took?

RxJS Observable of Actions

One idea that's been proposed as an alternative to Object.observe() is the Observable proposal and RxJS. Observables are a completely different construct in that they do not listen for changes on objects. They're just streams of objects that you can filter(), map(), merge(), etc. Observables would not serve as the basis for a change-tracking system for an ODM, but they work great as a middleware mechanism for action streams.

Monogram collections have an action$ stream that represents every function call you make on that collection:

const { connect } = require('monogram');

test().catch(error => console.error(error.stack));

async function test() {
  const db = await connect('mongodb://localhost:27017/test');

  const Test = db.collection('Test');

  Test.action$.subscribe(action => {
    /* Prints the below object. Note the `promise` property!
     * { _id: 5972314216958974eeadf4e3,
     *   params: [ { hello: 'world', _id: 5972314216958974eeadf4e4 } ],
     *   collection: 'Test',
     *   name: 'insertOne',
     *   chained: [],
     *   promise: Promise { <pending> } }
     */
    console.log(action);
  });

  await Test.insertOne({ hello: 'world' });
}

Monogram added promise property to the action object. This promise is the return value of the native MongoDB driver's insertOne() function. You can use this promise in the subscribe callback to print out each database operation and how long it took.

const { connect } = require('monogram');

test().catch(error => console.error(error.stack));

async function test() {
  const db = await connect('mongodb://localhost:27017/test');

  const Test = db.collection('Test');

  Test.action$.subscribe(({ collection, name, promise }) => {
    const start = Date.now();
    // Prints "Test.insertOne() 12ms"
    promise.
      then(() => { console.log(`${collection}.${name}() ${Date.now() - start}ms`); }).
      catch(() => { console.log(`${collection}.${name}() ERROR`); });
  });

  await Test.insertOne({ hello: 'world' });
}

Actions are such a powerful abstraction because they always have the same form. In mongoose, middleware comes in a motley crew of forms. It's easy to add a pre('save') hook, but it's hard to log every single operation or have common logic that works with both pre('save') and pre('find'). In monogram, every action has the same properties and goes through the same pipeline, so actions are easier to instrument. And, because they're easier to instrument, they're also easier to mutate.

Mutating Actions

You might wonder why there's the pre() middleware function and an observable of actions. The reason is similar to the difference between hot and cold observables. Once an action is assigned a promise property, you can no longer mutate or cancel that action, it's already in flight. You can mutate the promise and the action object, but that won't affect what gets sent to the database. In pre(), however, you can do whatever you want to the action object. For example, you can disallow deleting documents:

const { connect } = require('monogram');

test().catch(error => console.error(error.stack));

async function test() {
  const db = await connect('mongodb://localhost:27017/test');

  const Test = db.collection('Test');

  Test.pre(/delete/, action => {
    throw new Error('Deletes are not allowed!');
  });

  // Throws an error "Deletes are not allowed!"
  await Test.deleteOne({ hello: 'world' });
}

How about if you wanted to transparently support soft deletes rather than throw an error if a developer tries to deleteOne()? That's easy too. You can transform the action object in pre() middleware. You can change the parameters and even the function that will get called:

async function test() {
  const db = await connect('mongodb://localhost:27017/test');

  const Test = db.collection('Test');

  Test.pre(/delete/, action => {
    // deleteOne -> updateOne, deleteMany -> updateMany
    action.name = action.name.replace(/delete/, 'update');
    if (action.params.length < 1) {
      action.params.push({});
    }
    // Deletes transparently become updates that set an `isDeleted` flag
    action.params.push({ $set: { isDeleted: true } });
    return action;
  });

  // Log actions to the console for educational purposes
  Test.action$.subscribe(action => {
    console.log(inspect(action, { depth: 10 }));
  });

  /* Prints:
   * { _id: 597242530620367e5dfa4afa,
   *   params: [ { hello: 'world', _id: 597242530620367e5dfa4afb } ],
   *   collection: 'Test',
   *   name: 'insertOne',
   *   chained: [],
   *   promise: Promise { <pending> } }
   */
  await Test.insertOne({ hello: 'world' });

  /* This becomes an `updateOne()` under the hood!
   * { _id: 597242540620367e5dfa4afc,
   *   params: [ {}, { '$set': { isDeleted: true } } ],
   *   collection: 'Test',
   *   name: 'updateOne',
   *   chained: [],
   *   promise: Promise { <pending> } }
   */
  await Test.deleteOne();
}

What about backing up deleted documents to a separate collection? This has always been tricky since MongoDB deprecated copyTo(), but monogram makes this easy too:

async function test() {
  const db = await connect('mongodb://localhost:27017/test');

  const Test = db.collection('Test');
  const TestBackup = db.collection('TestBackup');

  Test.pre('deleteOne', async function (action) {
    const doc = await Test.findOne(action.params[0]);
    if (doc) {
      await TestBackup.insertOne(doc);
    }
    return action;
  });

  Test.action$.subscribe(({ collection, name }) => {
    console.log(`${collection}.${name}()`);
  });

  TestBackup.action$.subscribe(({ collection, name }) => {
    console.log(`${collection}.${name}()`);
  });

  // Test.insertOne()
  await Test.insertOne({ hello: 'world' });
  /*
   * Test.findOne()
   * TestBackup.insertOne()
   * Test.deleteOne()
   */
  await Test.deleteOne();
}

Moving On

Monogram is a potent abstraction for both advanced and novice users of MongoDB and Node.js. From logging to soft deletes to enforcing best practices to distributed locking, monogram makes instrumenting logic and handling cross-cutting concerns easy. Give it a shot and open up any issues you find on GitHub.