What Motivates You?

This is a break from my VMware Storage Series based on some recent conversations I have been having. I was recently given a reason to consider my professional career, what direction do I want to go, what is important, and what motivates me.

The conversation started a few weeks ago, and a friend, someone I had worked with in a previous life was discussing why he had joined a startup company with a bright future. He told me he was motivated by having fun. Money and the people he worked with were secondary, but having fun was really what he was all about.

I started discussing the option of joining him at the startup, with my friends, my wife, and a number of people in the industry I have a great deal of respect for. As I began to explore, a reoccurring theme kept coming up. I didn’t have a clear picture of what motivated me.

After much debating and discussing, I have come to realize that what motivates me is learning, teaching, and succeeding. I love winning, moving the ball forward as it were. I enjoy speaking in front of customers and perspective customers, and I love to get people excited about technology, and helping them see how it can help them.

I firmly believe this is a critical turning point in my career. I have chosen to stay where I am, I like my job, and my manager has assembled a team that is second to no one. I am doing my best to be a good contributing member, and I feel like there is plenty of room for growth as long as I put in the effort. It is the right decision for me at this point in my life.

So now that I have gone on this little journey of discovery, I would challange you, pause for a moment, take a look at your life, and ask yourself. “What motivates me?”

What Motivates You?

VMware Storage Part 5: Storage Networking Overview

Storage networking is an interesting discussion. This is usually pretty dependent on your storage vendor, or your personal preferences, but this is often also a misunderstood issue. When I was new to storage, we used to joke about iSCSI, real men used Fibre Channel. File storage like NFS was someone else’s problem, I was only interested in block, fibre channel storage. In a VMware environment, it becomes increasingly clear that we need to examine all types of storage networking to make sure that we have the right fit. Now I did talk a bit about this topic in previous posts on SAN and NAS, but I feel it is worth discussing in greater detail.

To start, VMware fully supports NFS, linux Network File System, and this is a perfectly logical way to handle storage in this environment. It is simple, it works much like a traditional file system, it is easy to grow, and is generally thin provisioned by default. There is no VMware File System, VMFS, to manage, the NFS server provides the file system. In cases where datastore sizes need to change often, and cases where redundancy is less important this might work. The main draw here at this point is simplicity of provisioning, and management. This can happen over 1GbE and 10GbE, and this really has to be decided based on network bandwith needs. It can be done using a converged network, but should always be on a separate subnet from the rest of the network traffic.

The major downside of NFS is that it is sometimes treated by VMware as a second class citizen. Most times when VMware releases new features from a storage perspective they are released on block storage. NFS is usually not far behind, but it does take time. Another downside is a lack of multipathing. Now I will say there are ways to do multipathing with NFS in a VMware environment, but it is more complex and not always a standard. Finally NFS is heavily reliant on the NFS server. While there are some good systems out there, there is also a reason a majority of the largest deployments have opted to use block storage for VMware. It is more trusted, and gives more options for storage vendors.

iSCSI is another IP based protocol, supported in both 1GbE and 10GbE. This is a great protocol for small to mid sized environments. It is simple to configure, and runs on the traditional IP network, using traditional IP switches with a few minor modifications. iSCSI is attractive because it generally keeps costs down. It also can run over a converged network, but should be isolated by VLAN’s at a minimum, and dependent upon your storage vendor may take advantage of multiple paths for redundancy or performance. Modern storage arrays are moving away from 1GbE in favor of 10GbE for iSCSI, which is something to consider if looking at this as an option.

The major downside of iSCSI is that it is so easy to deploy improperly, and is seldom designed correctly. Being an IP based protocol it is not purpose built for storage, so there is inherent latency. Jumbo frames and flow control, though widely debated, often time can have tremendous impact on the storage in a VMware environment. With 1GbE networking in larger environments, or 10GbE in a converged network, speed and complexity can be a factor. In a future post I will discuss storage network design in more details, but this should give a high level idea of some of the challanges.

Fibre Channel Networking, in the enterprise at least, is probably the most common protocol. The major advantage is this is dedicated. A fibre channel network is designed to transmit the SCSI commands between the storage and the host server. This becomes particularly important in a virtualized environment. Again more on this in a future post on storage network design. The major advantage here is that the latency is very low, and the switches are not having to handle any other traffic. In a virtualized environment, we also want to consider the number of servers per host we are using. In a traditional physical server if we have 2 fibre channel hba ports on the servers, and we virtualize 20 servers per host, we now have 10 times as many servers using the same bandwith. In this context, the low latency and lack of resource contention on the storage network is much more important.

The downside for Fibre Channel is cost and complexity. This requires dedicated specialized switches that cannot be used for anything else outside of storage. They are costly to deploy and to manage when compared to a typical IP network. They are also complex. They are often times outside the traditional network teams expertise so they will leave management to the storage team.

Finally there are of course Fibre Channel over Ethernet, FCOE, and Serial Attached SCSI, SAS, but these are less common in most environments and have not as of yet gained wide adoption.

At the end of the day these all get us to the same place, and all have their merrits. There are pros and cons to each, and I will talk more about the actual network design soon. It is good to know about each and where it may or may not be a good fit in a particular environment.

VMware Storage Part 5: Storage Networking Overview

VMware Storage Part 4 1/2: VSAN revisited

No sooner do I get a post up about VSAN and how I don’t think this is a major storage play, but rather a lower end play, than I see this on twitter…


Now I want to say, if your not reading guys like Duncan and Scott to name just a very few, then you are missing a majority of what is happening in the virtualization world. With the recent hires at VMware technical marketing, the future is anything but boring. That being said, I love speculating, and I do realize this is typical for VMware to push the envelope, release new and innovative products and then make them even more awesome.

I would have to say though this is significant if it means what it seems to. There are so many amazing products on the market now from a storage perspective alone, that VMware would be missing a huge segment if they didn’t work on this. I do stand by my thoughts though, this doesn’t kill competition, it just makes the rest of us work harder to innovate.

Competition is never a bad thing, it can bring out the best in us. I look forward to seeing what this means for those of us who have invested heavily in the future of VMware, from a career standpoint, and those who are using the products. What an exciting time to be alive and in technology.

VMware Storage Part 4 1/2: VSAN revisited


Going through the VMware storage options I would be remiss if I did not talk about VSAN. ┬áVSAN is simply VMware’s next way of helping to further improve the Software Defined Datacenter. ┬áTo begin with though, it is important to understand a little about what VMware is trying to accomplish. 2 years ago, on gigaom, an interesting article came through on VMware’s slow and steady attack on storage. This was following the release of their less than stellar Virtual Storage Appliance, VSA. At the time I took exception with this. Of course VMware would never want to get into the storage business, they are a software company. Then came VSAN.

I still do not believe VMware intends to completely own the storage market, but they are certainly changing the game. Now I work for HP, and I remember when server virtualization started to take off, we thought the server was going to become irrelevant, we would just use some cheap whitebox server. Fortunately we at HP realized that we had to step up our game. As usual the server engineering team worked with our alliance partners and built even better products designed around virtualization to give higher virtualization density, and higher performance. I equate this latest storage product to the same thing. It will certainly capture certain market segments, but it is not a threat to the core storage business of the larger storage vendors.

With that said, just what is VSAN? The concept behind VSAN is actually an old one come again. We have been doing scale out object storage for some time in this industry. VSAN simply moves this into the hypervisor stack. The requirements are pretty simple, you need a minimum of 3 host servers running vSphere 5.5 each with an SSD and at least one SAS or SATA drive, HDD. The requirements are well documented so I don’t want to get into those details, but this is enough to get started.

Conceptually, the SSD becomes a cache to accelerate the reads and writes to the drives. The HDDs are used for storage, and replicas are kept based on the rules, generally at least 2 copies of the data on separate hosts.

The setup is pretty simple, and there are hands on labs available online with VMware. It is also quite simple to setup using VMware Workstation running vSphere 5.5 for labs.

This scales currently, in Beta v1, to 8 hosts, so this is not going to be a massive system, more of an SMB environment or a lab system. It also introduces some interesting challenges on the server and network side. On the server, there is pretty limited official support since the raid controller has to enable pass through. There is no raid since this is an object store, data protection is accomplished through multiple copies of the data. On the network side, this is challenging because we are copying data between hosts to retain consistancy. This generally, in my mind, means that the era of running VMware on 1GbE networks is probably nearing an end.

At the end of the day, VMware has a number of use cases. This is a Beta v1 product, I am nervous about running it in production just yet. Many of their use cases are around high performance workloads, VDI for example, where user experience can make or break a project. I do think that this is an exceptional way of creating shared storage in a lab, and gives us many new ways to work in a lab environment.

As to the future of VMware storage, traditional storage, and our industry, I think this is the beginning of the next big thing. I will be discussing HP’s own software defined storage soon, as well as our traditional storage platforms in a VMware context. I don’t see VSAN as a threat, but rather as a call to action on our part to make our products better and continue to innovate. I will personally use VSAN in my lab along side HP StoreVirtual, different use cases, and fun to test.


VMware Storage Part 3: NAS

Nas is an interesting topic when it comes to VMware. This is often a religious debate, many users of products that are better at file storage than block love to talk about using VMware on NAS. Now there is really nothing wrong with that, NAS is a great medium for VMware storage.

To start with, NAS, Network Attached Storage, is nothing more than allowing multiple client machines to share storage. Likely your PC has a shared drive, probably several, your public or departmental share is on a NAS using the windows SMB, Server Message Block, protocol.

In a VMware environment, we use the NFS, Network File System, a linux based protocol to connect the servers to the storage. Remember with VMware we want to use shared storage for High Availability and load distribution. The advantage to using NFS really comes down to simplicity. When we use NFS storage in VMware what we are doing is just creating a file rather than writing blocks. This was an early attraction when block based storage was not able to keep up with the writes coming from VMware. Since it was writing to an open file, there was no concept of writing and committing data, it was all just writing. This hasn’t changed, but block storage has gotten significantly better, but that is for another post.

The simplicity also comes from the ability to expand simply. If you have the space on the NFS server you can just grow the NFS share, no extents (joining multiple logical volumes together), just grow the file system. Really not much too it, and very simple to manage you also don’t worry about the file system. In a block based system you create a file system based on VMFS, the VMware File System, on NFS, it is it’s own file system, you are just a file living there.

So the plus side is this is pretty simple, and the performance is now about the same from block to file, so what is the downside?

The biggest issue on my side is multipathing. There are ways to get around this, mostly proprietary, and with proper networking you can actually use nic bonding to give you a sense of multiple paths, but this requires some planning outside of the VMware environment, and can be challanging. On that point you are bound by your IP network, remember this doesn’t run over Fiber Channel, so if you go this direction, you better be on a rock solid network.

The other major downside, from my perspective, is VMware typically releases the features first to the block systems and then to NAS. Now again this gap is closing, but it is there. If you are like me, you update your iPhone to the latest code the min it is released and the servers stop crashing. In my lab, I run bleeding edge code, and I am all about the coolest and flashy, so this is important to me.

At the end of the day, NAS is great in a VMware environment. I personally like to have both options available, and would never tell someone they are wrong for going either direction, just make sure you are able to justify your decision.

VMware Storage Part 3: NAS

VMware Storage Part 2: SAN

Continuing on with the theme from last week. Again this is just to get things started, I do intend to dig into some of these areas much deeper, and discuss specific products, but to assume that everyone knows the basics would be in opposition to my desire to make technology, specifically virtualizaiton, something we can all understand and work with.

I was first introduced to the concept of a SAN with the Apple X-Raid in 2004 by a software developer I worked with. We were at a small software startup in Sacramento, CA, and the concept seemed outrageous to me. Shortly after that I ended up moving to a Casino where I was handed a SAN and the VMware ISO images. I quickly learned the value of shared storage.

The concept behind a SAN is that rather than the islands of storage we talked about in part 1, we can logically divide a large pool of storage between multiple servers. This is important in a VMware environment to enable things such as High Availability and Load Balancing between hosts. Since all hosts have access to all shared storage (SAN), a Virtual Machine may reside on any host.

A critical design point when using a SAN in any environment, but especially in VMware, is multipathing. This is simply having more than one connection from the host server to the shared storage. This becomes particularly critical in a VMware environment. Remember we are dealing with consolidation, so I may be moving 5,10, or more workloads to a single VMware host. Not only does this increase risk but also the load carried by the storage connections. This is where your SAN vendor’s controller design can help or hurt you, but that is a topic for another day.

SAN’s come in many flavors, but the connectivity methods are generally iSCSI, Fiber Channel, and Fiber Channel Over Ethernet. Each of these has it’s own advantages and disadvantages. What I have found is generally speaking this is largely dependent on the environment. For smaller customers, iSCSI is often perfectly acceptable, and can provide a familiar medium for the networking team. In larger environments, Fiber Channel is often preferable since it offers a simple and low latency network which is designed to do one thing and one thing only.

One closing thought on storage networking, it is important to consider line speed. With the release of 16G Fiber Channel, and 10GbE becoming more affordable, it is often wise to step up and pay for the fastest storage network you can afford. As Solid State Drives continue to gain market share we are seeing more and more storage networks become saturated. Many storage array vendors are dropping support for slower speeds, 1GbE iSCSI in particular. Always wise to prevent bottlenecks wherever possible even if it does cost a little more up front.

VMware Storage Part 2: SAN

VMware Storage Part 1: Introduction & Local Storage

One of my favorite radio talk hosts talks about the importance of having the “heart of a teacher”. When I started out in IT, I thought I wanted to be in tech support, teach others how to use their computers, and how to make the technology work for them. I have come to realize that while I enjoy helping others, I prefer to talk about concepts, and help them understand storage and virtualization. I am going to spend the next several posts going through some of the VMware storage concepts, in what to many may seem simple terms, but many of the people I talk to do not have a solid understanding, so I think it is always wise to level set, to start from a common point as it were. While there are many blogs out there with some incredibly technical content on this, many well written and helpful, I thought I would give this my own slant in an attempt to help some of the people I interact with and meet new ones. Feedback is appreciated, and I am always open to suggestions for new topics.

VMware in general is all about abstraction. With compute we put the software layer between the physical hardware and the operating system. This enables us to have portable servers, and to consolidate many workloads on to a smaller physical footprint. When we think about this from the storage side of things, it is not so much different. If we think about VMware creating a container to hold many servers, then a datastore, storage presented to VMware to hold Virtual Machines can be considered a container to store the hard drives and configuration files that make up a Virtual Machine. This storage is presented as one or more logical drives, datastores in VMware terms, up to 64TB in size. The reason behind sizing a datastore will be covered later, and is certainly open for discussion, but it is enough to know for now that we create a datastore from logical disk space.

When creating a Virtual Machine, VMware will ask you how much space you want, and which datastore you want to place it on. This will again be covered in a future post about design, but it is important to note, a datastore can contain multiple Virtual Machines, much like a VMware Host, physical machine running VMware, can contain multiple Virtual Machines.

Each VMware Host machine, provided it contains local hard drives, will have a datastore called “Local Datastore” or something similar. This is not a bad thing, it can be useful for Virtual Machines which you do not want to be able to move, but it is limited in that shared storage is required for high availability and resource distribution. With the release of VSAN in vSphere 5.5, as well as many Virtual Storage Appliance, VSA products, this can be used as shared storage as well, but more on that later.

To wrap up, storage is one of the more critical aspects to virtualization success. There are many more features to cover, I will be explaining many of the VMware features as well as where each different HP storage product may make sense as well as some reasons why I personally would choose HP over competitors products. Stay tuned and please let me know if there are other topics I should be covering, or if there is more detail needed.

VMware Storage Part 1: Introduction & Local Storage