MongoDB World’17 #MDBW17

MongoDB World 2017 (MDBW17) was amazing. You know, it is a conference with a marketing vies, but there are conferences and conferences.

MongoDB has been innovating on the database market since it has been born, but they made crystal clear at MDBW17 that a new milestone was achieved.

I’m pretty sure that the features they released and the ones that they are about to release on MongoDB 3.6 will change the way we create and deploy applications.

Let’s see if I can summarize all the cool stuff on this post.

MongoDB Advocacy Summit

Being part of the MongoDB Advocacy community is an awesome thing. We have been always in touch with each other virtually, and some of them have already met in person, but this year was special for me. I have the opportunity to meet my Advocate buddies in person.

The day was divided in two parts:

  • During the morning: roadmap overview presented by Eliot Horowitz(CTO and Co-founder), Dan Pasette (VP of Core Engineering), Sahir Azam (VP of Cloud Products and GTM), Michael Cahill (Director of MongoDB Storage Engines) and Asya Kamsky (Lead Product Manager)
  • Afternoon: unconference format. Here is how it works. People attending the summit either suggests or vote on subjects that they would like to discuss. Then, the three most voted subjects are spread across the room and people can choose which subject they want to join on a discussion creating three discussion groups. Each group has a volunteer to take notes and at the end each group summarizes the takeaways of each subject.

It was a very productive day. I’ve learned a lot from other Advocates.

Francesca Krihely, Jay Destro and Meghan Gill made an outstanding work putting the Advocate community together on Monday. Thanks guys! It was amazing.

mdbadvocacysummit01

Left to right: Logwriter, Jay Destro and Francesca Krihely

A New World of Possibilities Unleashed by MongoDB

Are you ready? Tons of new features and services were announced during the Keynote session.

Two announcements were shining on stage: MongoDB Stitch and MongoDB Charts.

MongoDB Stitch is a Backend-as-a-Service offer that promises to let developers focus on building application instead of service integration and backend infrastructure.

mdbstitch02

MongoDB Stitch Architecture Diagram

Described by Eliot Horowitz as a REST API for MongoDB and a service composition tool anywhere you want.

Definitely a candidate for a future post as soon as I learn it in more depth.

MongoDB Charts is a BI tool, which connects on ATLAS and allows you to explore and visualize your data using charts. My understanding was that is only available for ATLAS, but I have to double check that.

mdbcharts02

MongoDB Charts Dashboard

MongoDB ATLAS has completed one year since it was launched at MDBW16 and it’s been a successful DBaaS solution.

MongoDB 3.6 will bring interesting improvements such as retryable writes, notification API, better array manipulation, improvements in the aggregation framework and schema validation using a JSON.

NetApp Booth

NetApp was a platinum sponsor for MDBW17. You might be question yourself: “What a data storage company like NetApp is doing at MDBW17?”.

The first thing it might come to your mind would be: “Databases have to persist data on disk. NetApp sells data storage systems. Bingo!”.

Well, NetApp isn’t only a data storage company. We’ve reinvented ourselves as a company and became a data management company. Our solutions are much more than data storage systems. Click here to get more information about NetApp or if you have any specific questions, leave comment on this post and I will get back to you.

netappbooth01

NetApp Booth at MDBW17

Thanks for reading it and let me know if you have any questions.

See you next post!

Logwriter

 

It might happen to you anytime: Be ready! profiling, analyzing and creating indexes to support your App

Hi Everyone,

Logwriter here! How are you?

Usually when you are a Developer/DBA, you probably have a tied control of what queries your application does against the database and you kinda know which indexes will be necessary to support the queries.

But I bet that a lot of us working in the enterprise companies have the Developer/DBA role splitted because of the organization. So, you’re a DBA or you’re a Developer.

When that is the case and you’re the DBA, doesn’t matter how many times you’ve attended to SCRUM meetings with the Developer team, they will create a new feature that is querying data without an index. If this situation has never happened to you:

  • You are lucky guy, but there is always a first time for everything in life
  • Feel free to stop reading here if you don’t believe that it will never happen with you

It’s a wonderful day, you’re working in your comfy chair and all of a sudden you get a ticket about performance issues in the application forwarded by the application team to you. Of course my friend, the application team will always blame the database (they are supposed to do that, is their mission in life).

Users are complaining about slowness in the app when accessing/filtering cities per population sizes.

How you will find out what is happening?

One approach could be turning on database profiling:

> db.setProfilingLevel(1,10)
{ "was" : 0, "slowms" : 100, "ok" : 1 }

Right after the execution of the above command, you have a new collection in your database known as “system.profile”.

> show collections
system.profile
zips

Any operation that exceeds 10ms will be recorded in the system.profile collection.

note1: let’s assume that 10ms is the ceiling of acceptable latency for my app. Just because I don’t have a humongous database here.

I’m a lucky guy and the issue just happened and was recorded in the system.profile collection. So, analyzing the output:

> db.system.profile.find().pretty()
{
 "op" : "getmore",
 "ns" : "postal.zips",
 "query" : {
 "getMore" : NumberLong("25579730777"),
 "collection" : "zips"
 },
 "originatingCommand" : {
 "find" : "zips",
 "filter" : {
 "pop" : {
 "$gt" : 12000
 }
 }
 },
 "cursorid" : 25579730777,
 "keysExamined" : 0,
 "docsExamined" : 29178,
 "cursorExhausted" : true,
 "numYield" : 228,
 "locks" : {
 "Global" : {
 "acquireCount" : {
 "r" : NumberLong(458)
 }
 },
 "Database" : {
 "acquireCount" : {
 "r" : NumberLong(229)
 }
 },
 "Collection" : {
 "acquireCount" : {
 "r" : NumberLong(229)
 }
 }
 },
 "nreturned" : 6751,
 "responseLength" : 681209,
 "protocol" : "op_command",
 "millis" : 12,
 "planSummary" : "COLLSCAN",
 "execStats" : {
 "stage" : "COLLSCAN",
 "filter" : {
 "pop" : {
 "$gt" : 12000
 }
 },
 "nReturned" : 6852,
 "executionTimeMillisEstimate" : 11,
 "works" : 29469,
 "advanced" : 6852,
 "needTime" : 22616,
 "needYield" : 0,
 "saveState" : 231,
 "restoreState" : 231,
 "isEOF" : 1,
 "invalidates" : 0,
 "direction" : "forward",
 "docsExamined" : 29467
 },
 "ts" : ISODate("2017-04-12T00:39:53.808Z"),
 "client" : "127.0.0.1",
 "appName" : "MongoDB Shell",
 "allUsers" : [ ],
 "user" : ""
}
> 

I highlighted the fields that I believe that are important for this troubleshooting.

Q: Which collection has been reported with an operation that exceeds 
the 10ms threshold?
A: zips.
"ns": "postal.zips"
Q: Which operation has exceeded the 10ms threshold? 
A: getmore.
"op": "getmore"
Q: Which command has been executed?
A: find command, using a filter where the field pop should be greater
than 12000.
"originatingCommand" : {
 "find" : "zips",
 "filter" : {
 "pop" : {
 "$gt" : 12000
 }
Q: How long was the execution of this command?
A: 12ms.
"millis" : 12
Q: Was this query supported by an index?
A: No, it wasn't. The query planner has pointed that this query is a
collection scan. 
"planSummary" : "COLLSCAN"
Q: How many documents were examined and how many were returned?
A: All documents in the collection (29,467) were examined and 
6,852 were returned.
"docsExamined" : 29467
"nReturned" : 6852

Since we have what we were looking for let’s disable the database profiling:

> db.setProfilingLevel(0)
{ "was" : 1, "slowms" : 10, "ok" : 1 }
> db.system.profile.drop()
true
>

OK, there is one treat here: COLLSCAN. How do we avoid a collection scan? Creating an index to support the query.

The application is executing a query filtering on the field “pop”, so let’s create an index to support this query:

> db.zips.createIndex( { "pop": 1 } )
{
 "createdCollectionAutomatically" : false,
 "numIndexesBefore" : 1,
 "numIndexesAfter" : 2,
 "ok" : 1
}
>

Index has been created, but how can I check if the execution time has been improved by the index?

Let’s create an explainable object and try the same query as the application:

> var execplan = db.zips.explain("executionStats")
> execplan.find( { "pop": { $gt: 12000 } } )
{
 "queryPlanner" : {
 "plannerVersion" : 1,
 "namespace" : "postal.zips",
 "indexFilterSet" : false,
 "parsedQuery" : {
 "pop" : {
 "$gt" : 12000
 }
 },
 "winningPlan" : {
 "stage" : "FETCH",
 "inputStage" : {
 "stage" : "IXSCAN",
 "keyPattern" : {
 "pop" : 1
 },
 "indexName" : "pop_1",
 "isMultiKey" : false,
 "multiKeyPaths" : {
 "pop" : [ ]
 },
 "isUnique" : false,
 "isSparse" : false,
 "isPartial" : false,
 "indexVersion" : 2,
 "direction" : "forward",
 "indexBounds" : {
 "pop" : [
 "(12000.0, inf.0]"
 ]
 }
 }
 },
 "rejectedPlans" : [ ]
 },
 "executionStats" : {
 "executionSuccess" : true,
 "nReturned" : 6852,
 "executionTimeMillis" : 8,
 "totalKeysExamined" : 6852,
 "totalDocsExamined" : 6852,
 "executionStages" : {
 "stage" : "FETCH",
 "nReturned" : 6852,
 "executionTimeMillisEstimate" : 10,
 "works" : 6853,
 "advanced" : 6852,
 "needTime" : 0,
 "needYield" : 0,
 "saveState" : 53,
 "restoreState" : 53,
 "isEOF" : 1,
 "invalidates" : 0,
 "docsExamined" : 6852,
 "alreadyHasObj" : 0,
 "inputStage" : {
 "stage" : "IXSCAN",
 "nReturned" : 6852,
 "executionTimeMillisEstimate" : 10,
 "works" : 6853,
 "advanced" : 6852,
 "needTime" : 0,
 "needYield" : 0,
 "saveState" : 53,
 "restoreState" : 53,
 "isEOF" : 1,
 "invalidates" : 0,
 "keyPattern" : {
 "pop" : 1
 },
 "indexName" : "pop_1",
 "isMultiKey" : false,
 "multiKeyPaths" : {
 "pop" : [ ]
 },
 "isUnique" : false,
 "isSparse" : false,
 "isPartial" : false,
 "indexVersion" : 2,
 "direction" : "forward",
 "indexBounds" : {
 "pop" : [
 "(12000.0, inf.0]"
 ]
 },
 "keysExamined" : 6852,
 "seeks" : 1,
 "dupsTested" : 0,
 "dupsDropped" : 0,
 "seenInvalidated" : 0
 }
 }
 },
 "serverInfo" : {
 "host" : "brbrain",
 "port" : 27017,
 "version" : "3.4.0-rc3",
 "gitVersion" : "7d68067e5a6272bb463acc4e7a6c6a144148039c"
 },
 "ok" : 1
}
> 

Not going on too much detail here, but no more COLLSCANs, now the query is supported by the index ( “stage” : “IXSCAN”).

The number of examined and returned documents are the same, indicating a returned per examined documents rate of 1 which is an excellent thing to have. (“nReturned” : 6852, “totalDocsExamined” : 6852).

The execution time came down to 8ms (“executionTimeMillis” : 8).

Database profiling is an powerful tool that can be used to identify which indexes can be created in your collection to improve execution times.

It’s a simple example, but I believe that you got the idea, right? 🙂

Thanks for reading it and let me know if you have any questions.

See you next post!

Logwriter

MongoDB Data Protection on NetApp All Flash FAS

Hi All,

Logwriter here! How are you?

I just want to make a quick post to share 3 videos about MongoDB Data Protection on NetApp All Flash FAS (AFF).

I hope you can get an idea how quicker and easier your backup/restore process can be if you have your MongoDB database on NetApp.

Backup
Demonstrating how fast, easy and non-disruptive is the process of backup a MongoDB database using NetApp snapshots
Link: https://www.youtube.com/watch?v=tPl35B4KmFs&t

Restore
Demonstrating how to restore a ~700GB MongoDB database in seconds with SnapRestore and at the end applying the database operation log to bring the database to the most up-to-date position before the disaster.
Link: https://www.youtube.com/watch?v=EsFJ_tqOWXM

FlexClone
Customer is running MongoDB 3.2.11 on the production environment. He would like to test MongoDB 3.4.2 without upgrade the production system. Using flexclone to create a dev/test environment where customer will have the chance to test MongoDB 3.4.2
Link: https://www.youtube.com/watch?v=7oCsw1On3ts

Thanks for reading and see you next post!

Logwriter

Making NetApp ONTAP Deployment Easier

Hi Everyone,

Logwriter here! How are you?

It is another day in the office and I just got a new gear for my performance testing project. Every time I have a new project I need to deploy an ONTAP cluster. It means I have to create aggregates, volumes, LUNs, LIFs, igroups, etc. I also need to create a kind of documentation about how the storage was deployed for the performance testing project.

If it is a recurring task, why not automate it?

I’ve started a personal project that I’ve named as “NetApp Fast Deployment Tool”. Basically, you define your storage deployment in a JSON file and this tool will deploy the environment for you.

One of my current projects is an Oracle Performance Testing on a new NetApp AFF platform. It is an Oracle RAC deployment with 6-nodes, on the NetApp side I have 1 SVM, protocol is FCP,  8 FCP LIFs, 6 igroups (one per RAC node), 20 Volumes and 20 LUNs. This config was deployed in 1 minute.

At the end, my JSON file will work as a documentation about how the storage has been deployed.

You can see a video of the tool in action here. The storage deployment is for a MongoDB environment.

[ WARNING ]  — progressive metal rock as background song, turn down your speaker volume (or turn it up)

How to Install

  1. Download it from GitHub:
  2. Download the NetApp Manageability SDK (NMSDK) for all platforms from NetApp support web site:
  3. Uncompress the NMSDK and copy the python libs NaElement.py and NaServer.py to the netapp-fast-deploy lib directory:
    • cd netapp-manageability-sdk-5.6/lib/python/NetApp
    • cp NaElement.py <NETAPP_FAST_DEPLOY_DIR>/lib
    • cp NaServer.py <NETAPP_FAST_DEPLOY_DIR>/lib

How to Use it

You will find out the instructions about how to use this tool at the GitHub repository page.

Go Further, Faster

If you like it and start using it, I would like to ask you a favor. If you find a bug, please open an issue here. I’ve started this tool to improve my time on my daily tasks, but I’m completely open to add new features and fix possible bugs that you might find on it.

Thanks for reading it and let me know if you have any questions.

See you next post!

ONTAP and MongoDB Working Together – Episode 1

Hi All,

Logwriter here! How are you?

Let me start this post with a question:

How many times have you attended to a meeting where someone has demonstrated a product or solution that solves all issues in the world, but not yours?

I’ve attended to some of those meetings and I didn’t enjoy any second of meetings like that and I know how boring they can be.

With that in mind, I’ve decided to blog about real user issues. In case of MongoDB, I will get my user stories from the mongodb community. My sources will be: mongodb user group and mongodb advocacy hub.

Quick introduction about NetApp ONTAP for those that haven’t heard about it. ONTAP is the operating system that runs on NetApp All Flash FAS Series and FAS Series storage systems. You can find more information about here.

That said, let’s go for our first real user issue. Just to make it friendly, let’s use a JSON notation to describe it here:

{
subject”    : “Mongo dump and restore shows drastically different db.stats()”,
question” : “I run a mongodump of our database every night.When I did a restore to a totally separate replica set the sizes are drastically different. I’m gathering that this is probably normal, but looking for a little more confirmation from the community.”,
source”     : “https://groups.google.com/forum/#!topic/mongodb-user/8MF4Tku8wnI&#8221;
}

According to this user, his production database has 96 collections. He got a mongodump from the production database and ran a mongorestore in a different server. It turned out that after mongorestore was done, he ended up with only 95 collections restored.

There isn’t any log file from the server where the mongorestore was done, but regardless of the reason of why mongorestore didn’t bring all collections in the restore operation, if this environment was running on a NetApp AFF/FAS System your backups would be done using ONTAP Snapshots and a restore on a different server would be done using a FlexClone.

I’ve blogged before about NetApp Snap Creator Framework and backing up a MongoDB database, check it out here.

If you want to know how would be the restore process and how long it takes to restore a 1 TiB  database, I also have blogged about it check it out here.

By the way, NetApp will be presenting a session at MongoDB Europe16. Paul Mu, Technical Director at NetApp and Mohinder Toor, Business Development Executive EMEA, will be talking about Deploying MongoDB on NetApp Storage. Check the schedule here and attend to their session to learn how NetApp can help you to improve your MongoDB infrastructure.

Please, let me know if you have any questions and see you next post!

Thanks for reading it!

Logwriter

NetApp Simulator Lab

Hi All,

Logwriter here! How are you?

My friend Neil Anderson from FlackBox has done an amazing job putting together a guide to build a NetApp Lab using ONTAP Simulator.

netappsim_ebook

Neil has capture every screenshot of every step needed to have an up and running NetApp Simulator Lab and everything is documented in his ebook. The ebook is available for free and you can download it here.

Let me know if you have any questions and see you next post.

Thanks for reading it!

All the best,

Logwriter

MongoDB Deployment Tips on NetApp All-Flash FAS Systems – Part 4

featured_image_mongodb_series

Hey All,

Logwriter here! How are you?

Evaluating storage efficiency isn’t an easy task. In my opinion, the hardest part is to choose a dataset that best represents a real system.

For example, I wouldn’t say that measuring storage savings from YCSB (Yahoo Cloud Serving Benchmark tool) dataset is the best thing to do, unless your data set looks like an YCSB data set.

The Data Set

For my testing I’ve decided to use a public data set. I found a public data set from Reddit. It’s a data set with public comments from May and June 2016. It is a single collection with 131,079,747 documents. Figure 1 shows a sample document and its fields.

part4_doc_sample

Figure 1. Sample document

The data was ingested to MongoDB using the command: mongoimport

MongoDB Cluster Topology

I have a MongoDB cluster made of a single Replica Set. My Replica Set has 3 members: 1 Primary, 1 Secondary and 1 Arbiter.

It means that the Primary has a copy of the database and he handles requests of reads/writes to it.

Secondary has also a copy of the database that is sent by the primary (replication).

Arbiter is a member that helps to elect a new primary in case of the failure of the current primary.

part4_mdb_cluster_diagram

Figure 2. MongoDB Cluster Topology

WiredTiger (WT) Compression

WiredTiger (WT) is the default storage engine for MongoDB 3.2. It’s completely different compared to MMAPv1. It delivers document level concurrency and it stores data in a more efficient way then MMAPv1 because it supports compression.

You can choose between two different compression libraries: snappy or zlib.  The default is the snappy lib.

Snappy is a data compression/decompression library written by Google. Further information about it here.

Zlib is a data compression/decompression library written by Jean-loup Gailly and Mark Adler. More info about it here.

All-Flash FAS (AFF) Storage Efficiency

ONTAP 9.0 data-reduction technologies, including inline compression, inline deduplication, and inline data compaction, can provide significant space savings. Savings can be further increased by using NetApp Snapshot® and NetApp FlexClone® technologies.

ONTAP 9.0 has introduced a new storage efficiency technique known as “inline data compaction”. It gets more than 1 logical block and if possible, stores them in a single physical 4KB block.

Let’s say your NetApp AFF running ONTAP 9.0 gets the following write requests:

part4_objpack_01

Figure 3. Example of Write Requests to the storage system.

How these write requests will be handled by inline compression and inline compaction? Take a look on Figure 4.

part4_objpack_02

Figure 4. ONTAP 9.0 inline compression and inline compaction.

In this example, after it breaks the requests in WAFL blocks (4KB blocks) it would need 11 blocks to store the data. Applying inline adaptive compression, the number of required blocks to store the data goes down to 8 blocks, and finally after inline data compaction the data is stored in 4 physical disk blocks.

Which Storage Efficiency Technology I Should Use on My Environment?

I hate to answer a question like that with: “It depends.”, but unfortunately this is the real answer. Let me walk you through the process that has made me to conclude that “it depends”.

I’ve done 24 different tests using the Reddit public data set.

MongoDB has a parameter to control the on-disk page max size. The larger the value, the better will be your read performance in a sequential workload. The smaller the value, the better will be your read performance for a random workload. By default, leaf_page_max is set to 32K.

If we turn off all the efficiencies (in MongoDB and ONTAP) the Reddit data set has used 171GB of space.

Keeping leaf_page_max at the default, turning Compression on at MongoDB and turning efficiency off at ONTAP, the Reddit data set has used 88GB.

part4_wtcon_oseoff

Figure 5. WiredTiger Compression ON and ONTAP Efficiency OFF

WiredTiger compression is an on-disk feature, so it maintain compressed data on disk, but it doesn’t maintain compression on memory. So, you spend CPU time to compress the data before to send it to disk and when you need to read the data on disk to populate your cache you need to decompress the block and then make it available on cache.

ONTAP has been built to provide storage efficiency without impact the performance of your database. So, if you want to let ONTAP working on saving space for you, we need to change the leaf_page_max from 32K to 4K.

 Changing that setting, the Reddit data set has used 130GB. It is a little bit less saving than MongoDB, but your application will experience a consistent and predictable low latency.

part4_wtoff_oseon

Figure 6. WiredTiger Compression off and ONTAP Efficiency on.

Please, let me know if you have any questions and see you next post!

Thanks for reading it!

Logwriter