MongoDB Deployment Tips on NetApp All-Flash FAS Systems – Part 2

featured_image_mongodb_series

Hey All,

Logwriter here! How are you?

This post became longer than I was expecting, so I’ve decided to change the agenda of this series a little bit. At part two, you will see:

  • WiredTiger and Its Write Path
  • NetApp Snap Creator Framework
  • Creating Your MongoDB Profile and Configuration on Your Snap Creator Server

That said, let’s start it… I hope you enjoy it !

NetApp Snap Creator is the tool you want to use for backup, restore and clone operations with your MongoDB database.

part2_mdb_snapcreator

Figure 1. Snap Creator Login screen.

But before to jump into how to use Snap Creator, let’s see how WiredTiger handles write operations.

WiredTiger and Its Write Path

WiredTiger uses a MultiVersion Concurrency Control (MVCC) to handle write transactions. At the beginning of a write operation, WiredTiger provides a point in time version (aka snapshot) of the data that resides on disk. The snapshot is a representation of that data in memory.

When is time to write to disk, WiredTiger writes all the snapshot’s dirty pages into the datafiles in a consistent way. This process is called checkpoint.

By default checkpoints are made every 60 seconds. It can be modified changing the parameter syncPeriodSecs at /etc/mongod.conf.

But checkpoints aren’t enough to guarantee the durability of the data. Between checkpoints MongoDB protects your data using journaling.

A journal record is a representation of the operations that are happening with your data. For example, a document update might result in changes to an index, so WiredTiger creates a single journal record that contains the update operation and the necessary changes to the index.

By default, MongoDB syncs WiredTiger buffers with disks in the following events:

  • Every 50 milliseconds;
  • If a write operation occurs with write concern of j: true, WiredTiger syncs the journal files immediately;

The journal sync interval can be modified using the option commitIntervalMs, the minimum value is 1 and maximum value is 500.

You’re probably asking yourself “Why must I know about WiredTiger snapshot, checkpoint and journaling?”

You should know about these things to understand what will be in your backup as soon as you create one.

Through the Write Concern(j: true or j:false), MongoDB let’s the application determines the data’s level of importance, or better to say the durability of the data.

So, if your application is sending write operations with j: true, when you take a snapshot for sure all the operations acknowledge by mongod will be in your backup. But if your application is sending write operations with j:false, it isn’t guarantee that all operations acknowledge by mongod will be in your snapshot, because you don’t know if you’re taking a snapshot before or after the WiredTiger buffers being flushed to the journal files.

Note: You might be thinking about db.fsyncLock(), but according to MongoDB it is disruptive to your database. It will make mongod lock everything for write operations and it might affect reads. The connection that has started the lock must keep open to send the unlock, otherwise a hard shutdown (process kill) has to be done on mongod to unlock your instance.

NetApp Snap Creator Framework

Snap Creator is an unified data protection platform for multiple applications, databases and operating systems. It helps NetApp’s customers to address the following challenges:

  • Achieve application-consistent data protection
  • Standardize and simplify backup, restore and disaster recovery tasks in any environment
  • Become cloud ready by reducing the complexity of unique business processes and the development and management of scripts, helping you to make your backup, restore, and disaster recovery tasks more agile.

Let’s take a look on how Snap Creator works. It’s made of two main components: Snap Creator Server (scServer) and Snap Creator Agents (scAgent). Figure 2 gives you a better view how it looks like:

part2_mdb_snapcreator_dp_arch

Figure 2. NetApp Snap Creator Framework Data Protection Architecture

You need a VM or bare metal server to run your scServer. This VM or server needs to be able to reach out to your ONTAP cluster because it will be responsible to send the API calls to backup/restore or clone your application.

The scAgent goes in your application’s server. So, in our case here, the scAgent is installed on my MongoDB primary and secondary servers.

Creating your MongoDB Profile and Configuration on Your Snap Creator Server

To connect to your Snap Creator Server you just need a browser and then you point to your scServer IP on port 8443. The first time you login you will see the screen shown by figure 3.

part2_mdb_sc_welcome_screen

Figure 3. Snap Creator Welcome screen.

After click on “OK”, you will be redirected to the profile creation screen shown here by figure 4.

part2_mdb_sc_new_profile

Figure 4. Creating a new profile.

At this point I’m going to create a profile called “production”. Here I’m using the profile as the representation of my MongoDB production environment. After click on “OK” you will redirected to the configuration wizard as shown by figure 5.

part2_mdb_cfg_wiz_01

Figure 5. Snap Creator Configuration Wizard.

You need to give a name to your config. The name of my config is PRD_Real_Time_Analytics_DB.

part2_mdb_cfg_wiz_02

Figure 6. Configuration Name.

Snap Creator doesn’t have a specific plug-in for MongoDB, so at this screen you will select “None” as your plug-in type.

part2_mdb_cfg_wiz_03

Figure 7. Plug-in type selection.

After that, you need to inform the name or IP address where your scAgent is installed. In our case here, I’ve installed my scAgent on my MongoDB replica members and I’m pointing to the name of my primary server in the configuration.

part2_mdb_cfg_wiz_04

Figure 8. Agent Configuration.

Now is time to let your scServer knows how to talk with your ONTAP cluster. At this point you need to inform which protocol do you want to use to connect to your ONTAP cluster/SVM. Here I’m using HTTP on port 80.

part2_mdb_cfg_wiz_05

Figure 9. Storage Connection Settings.

Now your scServer knows which protocol will be used to talk to ONTAP, but it still needs the connectivity and authentication information to send commands to it. Here you will inform your SVM/Cluster name or IP address and also username and password to connect to it.

part2_mdb_cfg_wiz_06

Figure 10. Controller and SVM Credentials.

scServer now connects to your SVM/Cluster and it will show a list of volumes (left panel) for that particular object. You need to select the volumes that contains your database and click on the “right arrow” button to move them to the right side as shown by figure 11. Then click on “save”.

part2_mdb_cfg_wiz_07

Figure 11. Selecting Your Database Volumes.

After that, Snap Creator shows you the list of volumes that you’ve selected and your credentials for your configuration. Then, click “next”.

part2_mdb_cfg_wiz_08

Figure 12. List of SVM/Cluster and Volumes will be part of your configuration.

Now is time to setup your backup and retention policies. For this example, we are setting up a daily backup policy with retention of 2 days.

part2_mdb_cfg_wiz_09

Figure 13. Backup and retention policy.

Here is the most critical step. As your database is spread across volumes (in my case I have 8 volumes), to backup it properly you need to take a consistency group snapshot. So, at this screen, don’t forget to click on “Consistency Group” check box before to click on “next”.

part2_mdb_cfg_wiz_10

Figure 14. Snapshot Details.

The next screen is about replication and remote backup, SnapMirror and SnapVault respectively. I’m not using any of these two options on my environment, so just click on “next”.

part2_mdb_cfg_wiz_11

Figure 15. SnapMirror and SnapVault.

If you have OnCommand Unified Manager in your environment and wants to let your scServer to send notifications for it, here is where you provide all the connection information to let that happen.

part2_mdb_cfg_wiz_12

Figure 16. NetApp OnCommand Unified Manager settings.

Then a Summary screen and the opportunity to review all your settings before to click on “Finish”.

part2_mdb_cfg_wiz_13

Figure 17. Configuration Summary.

After click on “Finish” your configuration is created and you are ready to create your first MongoDB backup.

part2_mdb_cfg_wiz_14

Figure 18. Configuration created successfully.

To create your first backup, select your configuration on the left panel. Click on “Actions” button and then “Backup”.

part2_mdb_action_backup

Figure 19. Backup your database.

On the next part of this series I will cover restore and cloning operations.

Thanks for reading it and let me know your opinion about this post leaving a comment.

A special thanks for Kevin Adistambha (Technical Service Engineer @ MongoDB). He had answered my questions about fsyncLock() and fsyncUnlock in the mongodb users forum.

See you soon,

Logwriter

References:

WiredTiger Internals

WiredTiger Snapshots and Checkpoints

NetApp Snap Creator Data Sheet

Advertisements

One thought on “MongoDB Deployment Tips on NetApp All-Flash FAS Systems – Part 2

  1. Pingback: ONTAP and MongoDB Working Together – Episode 1 | .:: logwriter ::.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s