Microsoft’s Azure Stack is an excellent toolset that allows enterprises to run a hybrid cloud right in their own datacenters, giving them additional cloud options.
But to really use it to its best advantage, IT pros should know the ins and outs of Azure Stack so they can use it within their business IT infrastructures to better manage, speed up and control their Azure cloud deployments and workloads.
A good place to start is with a primer on Azure Stack itself to give business users a broad look at what’s under the hood of their IT infrastructure.
Microsoft SQL Server product group announced that SQL Server, either virtual or bare metal, is fully supported on Storage Spaces Direct. The Exchange Team did not have a clear endorsement for Exchange on S2D and clearly still prefers that Exchange is deployed on physical servers with local JBODs using Exchange Database Availability Groups or that customers simply move to O365.
Here’s where we dive in and get dirty…but I promise by the end of my series, you will smiling like my friend here. I am planning a surprise with special guest bloggers. Stayed Tuned. Now one to the show…..
The NEW ReFS File System, Multi-Tier Volumes and Erasure Coding
Like S2D, the ReFS file system actually isn’t new either, they have been working on it for several releases now also. In Windows Server 2016, it finally drops the tech preview label and is now ready for production. And there is a lot of benefits… like volume creation doesn’t have to zero out the volume for 10 minutes like NTFS. It’s just a metadata operation that is effectively instantaneous now, I’m just going to focus on the couple of benefits that ReFS has for S2D.
For those not familiar Erasure coding (EC) and to prepare you for the next part, EC is a method of data protection in which data is broken into fragments, expanded and encoded with redundant data pieces and stored across a set of different locations.
The original goal of EC was to enable data that becomes corrupted at some point in the storage process to be reconstructed by using information about the data that’s stored elsewhere. Erasure codes are great, because of their ability to reduce the time and overhead required to reconstruct data. The drawback of erasure coding is that it can be more CPU-intensive, and that can translate into increased latency.
Now all that being said, classic erasure codes were designed and optimized more for communication, not for storage. Naively applying classic erasure codes in storage is okay, but is missing enormous efficiencies. Microsoft has developed their own erasure codes optimized for storage called Local Reconstruction Codes (LRC). I will cover this brieifly further down in the post.
Now back on to S2D…For data protection, S2D uses either 3-way mirroring or distributed parity with EC. Mirroring gives you great write performance, but only 33% data efficiency. EC gives you good data efficiency, but random write performance isn’t great for hot data. ReFS supports the ability to combine different disk tiers using different parity schemes in the same vDisk. This allows S2D to do real-time data tiering by writing new data to the mirror tier and then automatically rotating cold data out to the parity tier and applying the erasure code on data rotation.
It is important to note that ReFS does not currently support Deduplication. There was a question on this in every session and MSFT says that this is all the ReFS is currently focused on. So we’ll expect to see it land in ReFSv3. For now, customers can get dedupe with S2D by using NTFS. 🙁 Note if you only have two types of storage then the highest performing is used for the cache while the other type will be divided between performance and capacity with the different resiliency option (mirror vs parity) providing the performance/capacity difference between the tiers. If you only have one type of storage then the cache is disabled and the disks divided between performance and capacity like the previously mentioned case.
For non-Storage Spaces Direct only two tiers, of storage are supported like Windows Server 2012 R2, i.e. SSD and HDD, there is no cache. If you had NVMe storage that could be the “hot” tier while the rest of storage (SSD, HDD) could be the “cold” tier (you name the tiers whatever you want) but you cannot use three tiers. During Ignite 2016, Microsoft took many shots at VMware. Microsoft said that there’s a right way and a wrong way to do erasure coding. “When you do it the wrong way, performance sucks and you have to limit it to all-flash configurations.”
Microsoft research is using a new technique called “Local Reconstruction Codes”. It uses smaller groups within the vDisk that allows them to recover from failures much faster by not having to reconstruct data from across the entire pool. This combined with multi-tier volumes gives S2D good performance, even on hybrid systems. Sounds like a technology that I seen before. Hmmm..I wonder where……. 😉 Ok, that’s all for now. next up, Fault Tolerance and Multisite Replication with S2D….
Microsoft World Wide Partner Conference 2015 WPC is the largest event for Microsoft partners When it comes to meeting the right people in the right place, bigger is better. The Microsoft Worldwide Partner Conference (WPC) brings together over 15,000 attendees … Continue reading →
As a Microsoft Evangelist at Nutanix, I am always asked….”How would you monitor your Nutanix Infrastructure and can I use System Center suite. And my answer always is, “YES, with SCOM”….What is SCOM you ask? System Center Operations Manager (SCOM) is designed to be a monitoring tool for the datacenter. Think of a datacenter with multiple vendors representing multiple software and hardware products. Consequently, SCOM was developed to be extensible using the concept of management packs. Vendors typically develop one or more management packs for every product they want plugged into SCOM.
To facilitate these management packs, SCOM supports standard discovery and data collection mechanisms like SNMP, but also affords vendors the flexibility of native API driven data collection. Nutanix provides management packs that support using the Microsoft System Center Operations Manager (SCOM) to monitor a Nutanix cluster.
Nutanix SCOM Management Pack
The management packs collect information about software (cluster) elements through SNMP and hardware elements through ipmiutil (Intelligent Platform Management Interface Utility) and REST API calls and then package that information for SCOM to digest. Note: The Hardware Elements Management Pack leverages the ipmiutil program to gather information from Nutanix block for Fans, Power Supply and Temperature.
Nutanix provides two management packs:
Cluster Management Pack – This management pack collects information about software elements of a cluster including Controller VMs, storage pools, and containers.
Hardware Management Pack –This management pack collects information about hardware elements of a cluster including fans, power supplies, disks, and nodes.
Installing and configuring the management packs involves the following simple steps:
Install and configure SCOM on the Windows server system (if not installed) (will blog a post soon on this topic)
Open the IPMI-related ports (if not open). IPMI access is required for the hardware management pack
Install the Nutanix management packs
Configure the management packs using the SCOM discovery and template wizards
After the management packs have been installed and configured, you can use SCOM to monitor a variety of Nutanix objects including cluster, alert, and performance views as shown in examples below. Also, I check out this great video produced by pal @mcghem . He shows a great demo of the SCOM management pack…Kudo’s Mike….also, check out his blog.
Views and Objects Snapshots
Cluster Monitoring Snapshots
Cluster Performance Monitoring
Hardware Monitoring Snapshots
In the following diagram views, users can navigate to the components with failure.
Nutanix Objects Available for Monitoring via SCOM
The following provides an high level overview of Nutanix Cluster with Components:
The following sections describe Nutanix Cluster objects being monitored by this version of MPs:
Cluster
Monitored Element
Description
Version
Current cluster version. This is the nutanix-core package version expected on all the Controller VMs.
Status
Current Status of the cluster. This will usually be one of started or stopped
TotalStorageCapacity
Total storage capacity of the cluster
UsedStorageCapacity
Number of bytes of storage used on the cluster
Iops
For Performance: Cluster wide average IO operations per second
Latency
For Performance: Cluster wide average latency
CVM Resource Monitoring
Monitored Element
Description
ControllerVMId
Nutanix Controller VM Id
Memory
Total memory assigned to CVM
NumCpus
Total number of CPUs allocated to a CVM
Storage
Storage Pool
A storage pool is a group of physical disks from SSD and/or HDD tier.
Monitored Element
Description
PoolId
Storage pool id
PoolName
Name of the storage pool
TotalCapacity
Total capacity of the storage pool
Note: An alert if there is drop in capacity may indicate a bad disk.
UsedCapacity
Number of bytes used in the storage pool
Performance parameters:
Monitored Element
Description
IOPerSecond
Number of IO operations served per second from this storage pool.
AvgLatencyUsecs
Average IO latency for this storage pool in microseconds
Containers
A container is a subset of available storage within a storage pool. Containers hold the virtual disks (vDisks) used by virtual machines. Selecting a storage pool for a new container defines the physical disks where the vDisks will be stored.
Monitored Element
Description
ContainerId
Container id
ContainerName
Name of the container
TotalCapacity
Total capacity of the container
UsedCapacity
Number of bytes used in the container
Performance parameters:
Monitored Element
Description
IOPerSecond
Number of IO operations served per second from this container.
AvgLatencyUsecs
Average IO latency for this container in microseconds
Hardware Objects
Cluster
Monitored Element
Description
Discovery IP Address
IP address used for discovery of cluster
Cluster Incarnation ID
Unique ID of cluster
CPU Usage
CPU usage for all the nodes of cluster
Memory Usage
Memory usage for all the nodes of cluster
Node IP address
External IP address of Node
System Temperature
System Temperature
Disk
Monitored Element
Description
Disk State/health
Node state as returned by the PRISM [REST /hosts “state” attribute ]
Disk ID
ID assigned to the disk
Disk Name
Name of the disk (Full path where meta data stored)
Disk Serial Number
Serial number of disk
Hypervisor IP
Host OS IP where disk is installed
Tire Name
Disk Tire
CVM IP
Cluster VM IP which controls the disk
Total Capacity
Total Disk capacity
Used Capacity
Total Disk used
Online
If Disk is online or offline
Location
Disk location
Cluster Name
Disk cluster name
Discovery IP address
IP address through which Disk was discovered
Disk Status
Status of the disk
Node
Monitored Element
Description
Node State/health
Node state as returned by the PRISM [REST /hosts “state” attribute ]
Node IP address
External IP address of Node
IPMI Address
IPMI IP address of Node
Block Model
Hardware model of block
Block Serial Number
Serial number of block
CPU Usage %
CPU usage for Node
Memory Usage %
Memory usage for node
Fan Count
Total fans
Power Supply Count
Total Power supply
System Temperature
System Temperature
Fan
Monitored Element
Description
Fan number
Fan number
Fan speed
Fan speed in RPM
Power supply
Element
Description
Power supply number
Power supply number
Power supply status
Power supply status whether present or absent
If you would like to checkout the Nutanix management pack on your SCOM instance, please go to our portal to download the management pack and documentation. This management pack was development by our awesome engineering team @ Nutanix. Kudos to Yogi and team for a job well done!!! 😉 I hope I gave you a good feel for Nutanix monitoring using SCOM. As always, if you have any questions or comments, please leave below….
In this section I’ll cover how the various storage devices (SSD / HDD) are broken down, partitioned and utilized by the Nutanix platform. NOTE: All of the capacities used are in Base2 Gibibyte (GiB) instead of the Base10 Gigabyte (GB). Formatting of the drives with a filesystem and associated overheads has also been taken into account.
SSD Devices
SSD devices store a few key items which are explained in greater detail above:
Below we show an example of the storage breakdown for a Nutanix node’s SSD(s):
NOTE: The sizing for OpLog is done dynamically as of release 4.0.1 which will allow the extent store portion to grow dynamically. The values used are assuming a completely utilized OpLog. Graphics and proportions aren’t drawn to scale. When evaluating the Remaining GiB capacities do so from the top down. For example the Remaining GiB to be used for the OpLog calculation would be after Nutanix Home and Cassandra have been subtracted from the formatted SSD capacity. Most models ship with 1 or 2 SSDs, however the same construct applies for models shipping with more SSD devices. For example, if we apply this to an example 3060 or 6060 node which has 2 x 400GB SSDs this would give us 100GiB of OpLog, 40GiB of Content Cache and ~440GiB of Extent Store SSD capacity per node. Storage for Cassandra is a minimum reservation and may be larger depending on the quantity of data.
For a 3061 node which has 2 x 800GB SSDs this would give us 100GiB of OpLog, 40GiB of Content Cache and ~1.1TiB of Extent Store SSD capacity per node.
HDD Devices
Since HDD devices are primarily used for bulk storage, their breakdown is much simpler:
For example, if we apply this to an example 3060 node which has 4 x 1TB HDDs this would give us 80GiB reserved for Curator and ~3.4TiB of Extent Store HDD capacity per node. NOTE: the above values are accurate as of 4.0.1 and may vary by release. Next up, I figured we would look at some of the cool software technologies that run on our CVM (Controller Virtual Machine), next up Elastic Dedupe Engine.
Until next time, Rob
Like this:
LikeLoading...
This website uses cookies to improve your experience. We'll assume you're ok with this, but you can opt-out if you wish.AcceptReject
Privacy & Cookies Policy
Privacy Overview
This website uses cookies to improve your experience while you navigate through the website. Out of these, the cookies that are categorized as necessary are stored on your browser as they are essential for the working of basic functionalities of the website. We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may affect your browsing experience.
Necessary cookies are absolutely essential for the website to function properly. This category only includes cookies that ensures basic functionalities and security features of the website. These cookies do not store any personal information.
Any cookies that may not be particularly necessary for the website to function and is used specifically to collect user personal data via analytics, ads, other embedded contents are termed as non-necessary cookies. It is mandatory to procure user consent prior to running these cookies on your website.
Manage Cookie Consent
To provide the best experiences, we use technologies like cookies to store and/or access device information. Consenting to these technologies will allow us to process data such as browsing behavior or unique IDs on this site. Not consenting or withdrawing consent, may adversely affect certain features and functions.
Functional
Always active
The technical storage or access is strictly necessary for the legitimate purpose of enabling the use of a specific service explicitly requested by the subscriber or user, or for the sole purpose of carrying out the transmission of a communication over an electronic communications network.
Preferences
The technical storage or access is necessary for the legitimate purpose of storing preferences that are not requested by the subscriber or user.
Statistics
The technical storage or access that is used exclusively for statistical purposes.The technical storage or access that is used exclusively for anonymous statistical purposes. Without a subpoena, voluntary compliance on the part of your Internet Service Provider, or additional records from a third party, information stored or retrieved for this purpose alone cannot usually be used to identify you.
Marketing
The technical storage or access is required to create user profiles to send advertising, or to track the user on a website or across several websites for similar marketing purposes.
System Center Operations Manager (SCOM) is designed to be a monitoring tool for the datacenter. Think of a datacenter with multiple vendors representing multiple software and hardware products. Consequently, SCOM was developed to be extensible using the concept of management packs. Vendors typically develop one or more management packs for every product they want plugged into SCOM.
After the management packs have been installed and configured, you can use SCOM to monitor a variety of Nutanix objects including cluster, alert, and performance views as shown in examples below. Also, I check out this great video produced by pal @mcghem . He shows a great demo of the SCOM management pack…Kudo’s Mike….also, check out his blog.
You must be logged in to post a comment.