Overview

Mongo has a feature called the oplog, where inserts/updates/deletes are recorded for replication purposes (so that another Mongo node can read these operations and keep itself up-to-date). The oplog can be leveraged by other systems to do things like keep ElasticSearch in sync with Mongo.

Why?

Existing implementations of oplog tailing are hard to get started with and/or are not performant enough. We wanted to a better solution.

How Far Did We Get?

  • Cleaned up the existing code and made it very easy to get a working oplog tailer up and running
  • Code is now open source on Github.

What’s Next?

  • Get the jar on Maven Central
  • Use it for other things:
    • Smarter handling of indexing (instead of polluting the code with extra calls, we can watch the db for changes)
    • Performance/db operations auditing