Posted by & filed under Features.

ObjectRocket was built on the core premise of enabling simple and reliable scalability for MongoDB.

At it’s core, ObjectRocket utilizes MongoDB’s native scaling architecture called sharding. But it’s a cloud world, and in the cloud there are interesting things we can do when we have lots of compute at our disposal. Customers can start off small, and grow automatically. When users sign up, they instantly get a fully provisioned sharded cluster in whatever plan size they choose. All of the complicated sharding components are automatically setup, and there is zero configuration for a customer to complete. So scaling on ObjectRocket is seamless.

Here is how we do it.

At the core of the system is our automation agent called RocketScale™. This component is a back end job written in python/APScheduler. It’s job in life is to detect when a customer needs to grow, and then handles the process of adding a shard to the cluster. Each customer and instance pair gets an isolated APScheduler job queue for RocketScale to poll and analyze.

RocketScale works using some basic metadata instructions as it polls, one configurable by the customer. The setting named autoadd_shard_threshold is customer configurable, and instructs RocketScale when to add a new shard. Rocketscale polls our metadata for this threshold, then compares it to the actual disk usage of each instance. If the disk space used by the customer exceeds the threshold, a new shard is added. It’s just a simple bit of metadata as:
[javascript]
{autoadd_shard_threshold: 80}
[/javascript]
This setting is a configurable in the ObjectRocket UI under instances/{instance_name}. By default new instances are set to a default 80% threshold. We recommend sharding early and often to avoid any possible issues with the MongoDB balancer, and 80% is a nice compromise of space usage and ease of scaling. Setting RocketScale to 0 or leaving it empty disables this process.

It should be noted that MongoDB won’t be able to use multiple shards unless there are shard keys defined on the major collections. ObjectRocket customers must create these keys using our UI in order to instruct MongoDB how to logically split the data. Selecting shard keys can be time consuming and tricky, and we have some tools that are coming soon to help in this process. For today, users simply specify the shard key(s) they would like using the UI at https://app.objectrocket.com.

One interesting aspect of this process is that RocketScale always has a shard at hand to instantly add because of the way we provision shards. We pre-provision shards ahead of time in various plan sizes. Adding a shard is simply grabbing the next shard in line and assigning it to a customer. This is all done in the background and is very quick to complete. ObjectRocket manages the number of candidate shards in the queue and ensuring that we have the proper capacity to add for customers as needed.

RocketScale has a couple sanity checks built into it to ensure it’s doing the correct thing. For instance:

  • Checks the version of MongoDB to ensure it matches
  • Checks to ensure we haven’t added many shards in the last few minutes (runaway sharding!)
  • Sanity checks the instance is OK and usable

When a shard is added, the ObjectRocket notification system sends a message regarding the change. The message generally has the format of:

Hi ObjectRocket User,

Your ObjectRocket instance named 'foo' is getting low on storage space. 
RocketScale (our automatic scaling engine) will be adding a shard soon 
to horizontally scale your instance, and your monthly bill will increase 
accordingly.

The following collections don't have any shard keys defined, and 
won't make use of this additional shard:
---------
foo.mybigcollection_no_keys
foo.mysmallcollection_no_keys

Even if additional shards are added, if a collection without shard keys 
uses all of the remaining storage space on your instance, you could 
experience downtime. ObjectRocket strongly recommends you define 
shard keys for all of your collections.

You can add shard keys to your collections by following these steps:

1. Browse to https://app.objectrocket.com/instances/xxxxxxxx.
2. Click on the database name containing the collection without shard keys.
3. Click on the collection name that does not have a shard key.
4, Click on the 'Shard Key' tab.
5. Click on 'Add Shard Key'.

An excellent guide to selecting shard keys can be found at
http://docs.mongodb.org/manual/tutorial/choose-a-shard-key/ .

To silence future notifications for this instance, click the link below:

https://app.objectrocket.com/73h33bbbdd337754

If you have questions or issues, you can always ping us at:
support@objectrocket.com

Thanks!
The ObjectRocket Team
@objectrocket

In addition, ObjectRocket support also sees a message that the customer grew so we are in the loop in case the customer has questions.

Once a new shard is added, the MongoDB balancer is kicked off to balance the chunk data onto the new shard. In some cases the customer may have selected a balancer window and thus the balancer will not fire up until the specified window starts. Managing these settings is easy with our recent Balancer ease of use enhancements.

In addition to RocketScale there are user controllable mechanisms like the UI and the API where a shard can be added, but we will save those items for next time.

kgorman

Kenny has 15 years of experience with various database platforms behind some of the busiest websites in the world. He has had roles as Architect, Director, Manager, Developer, and DBA. He was a key member of the early teams that scaled Paypal and then eBay, and most recently was an early adopter of MongoDB using it for various large projects at Shutterfly. He has been an active MongoDB community member, speaker, MongoDB evangelist, and now Mongo Master.

More Posts - Website

Follow Me:
TwitterLinkedIn

Leave a Reply

  • (will not be published)