Clearspace Server Syndication Feed

Considerations and guidance when virtualizing key applications

2009-09-02T21:52:18Z

Application Virtualization is a key part of expanding the rationale for Virtualization. EMC, VMware and our partners are beginning to show with emerging technologies such as Fully Automated Storage Tiering (FAST), I/O dedupe, sub block copy and flash technology, that the final frontier - mission critical application Virtualization will become a reality very soon.

Successful application virtualization will have a knock on effect in terms of enhancing the overall cost savings of Virtualization as all the benefits of ease of management, consolidation, protection and efficient deployment can be extended to the entire infrastructure putting in place one set of standards and approaches to managing and provisioning the overall environment rather than a series of application centric silos

However, in the quest to virtualize the remaining mission critical application hold-outs, certain truths with remain in place. The first and most obvious of these truths is that the performance requirements of the application in the near term will remain the performance requirements of that application in the virtual world and consequently the spindles that drive that application in the physical world will need to drive it in the virtualized world.

Where this scenario will change in the future is around the understanding of the eligibility of certain services for new technologies such as Enterprise Flash drives and how emerging technologies such as FAST can ease the cost of providing performant storage for an application particularly when its performance requirements are cyclical. Cyclical performance requirements are characterised by that one transaction database batch job a month for which a company must provide a performance profile of expensive fibre channel disks but which is largely underutilised for the other 30 days at an unnecessary but very important cost to the business.

With the emergence of technologies such as FAST on the arrays initially and hopefully full DRS integration down the line, customers will be able to assign performance limits, maximums and reservations in the way that they currently do with memory and processing power. This should lead to a far more efficient overall storage consumption profile and an outcome that some in the industry are predicting could eventually lead to just two tiers of storage at the performance end of the spectrum with enterprise flash drives and at the capacity end with low speed large capacity SATA technology.

The second truth is not only should the individual application performance requirements being added into the mix of a particular virtualized environment be carefully profiled but that the overall performance and health of the infrastructure needs to be considered and architected for.

It is tempting with the large capacity currently being deployed in clusters and the historic lack of VMware administrator awareness of the underlying storage for organisations to look to deploy applications into environments where that application as a single entity will have a disproportionate impact on the overall environment's environments performance and possibly stability.

In these scenarios it is critically important to understand the resources that exist in the current environment and how the proposed application could impact those resources as well as how the application itself could be affected once inserted into a particular virtual infrastructure

To this end EMC in conjunction with VMware has done extensive testing of popular enterprise application such as Microsoft SharePoint, Exchange and SQL as well as SAP and Oracle to help customers understand how to intelligently and cost effectively scale applications on a virtualized platform. These documents not only explore storage technology but also the choice of connectivity protocol as well as the combination of array features with key VMware enabling technologies such as HA DRS and Fault Tolerance

MirrorView Insight for VMware - the next generation Site Recovery Adpater for VMware SRM that includes Failback

2009-09-02T08:01:24Z

I am writing this in the hope of drawing the VMware communities' attention to the next generation of of Site Recovery Adpaters, the storage interface into VMware's vCenter Site Recovery Manager. The SRA discussed here is the for the EMC CLARiiON Fibre Channel / iSCSI mid-tier array.

As this community understands, rather than simply being a series of scripts that calls the CLARiiON's replication capability on behalf of the SRM functionality, the latest SRA is an very useful application in itself and can be understood in its fullness as the MirrorView Insight for VMware application or MVIV

MVIV Shot 1

So what does it do? Fundamentally it does two things and does them well.

Firstly, it helps CLARiiON / VMware administrators, through a series of intuitive screens, navigate the process of designing and then verifying the MirrorView configuration as they plan to present it to SRM.

Secondly it enables those same administrators to automate the process of failing back that environment following on from a full, SRM driven, failover test - something that is only possible at the moment with SRM if done manually. That manual fail back is a process prone to error and painful consquences if not documented and carefully co-ordinated

As the MVIV 2 shot below demonstrates, the MVIV GUI allows the adminstrator to specify through free-text entry the two sites in his environment linked by SRM and CLARiiON replication - in my case, in the Lab, London and Palo Alto. These site identifiers are then populated throughout the application's various screens allowing the Administrator to keep clear where Virtual Machines belong in the environment

Now the importance of these designations is not to be underestimated in terms of keeping track of where Virtual Machines are once failover is intitiated - either by the MVIV tool itself or by SRM. The concept of clear site identities allows administrators to always know where a VM is and where it should be relative to its 'Production Site'. In many scenarios, the MVIV might be the only easy means of determining this - given a geographic split of administrative responsibilites

Once all the relevant CLARiiON and Virtual Center details are inputted and the application has completed its discovery of both the CLARiiON and the VMware environment, both VMware and CLARiiON administrators have access to a wealth of useful information

For example MVIV will show the Administrator - which LUNs are currently replicated by site, by MirrorView and whether those LUNs can be seen (as expected) by the ESX hosts at both sites on the Replicated LUNs on Array tab

MVIV Shot 3

Other useful information such as LUN name, LUN number, Size, MirrorView type and Consistency Group Membership are displayed

The second tab, the Replicated LUNs on ESX tab allows the Administrator to drill down into the MirrorView configuration and validate that the correct Source and Target LUNs are being seen by the respective hosts on the two sites as well as the characteristics of those LUNs including such useful information as whether the Mirrored LUNs have the mandatory SNAPshot assigned using the correct naming convention, consistency group membership etc.

MVIV Shot 4

The Third tab Datastores quickly shows the administrator which datastores are currently replicated via MirrorView in a compliant fashion for MVIV. Compliant, for example, in the fact that if the datastore spanned multiple LUNs and only some of those LUNs were mirrored, the screen would show that Datastore as non-compliant and with a warning icon.

It will also show if the Virtual Machines within the datastore are replicated. This again is important in scenarios where a Virtual Machine may be composed of VMDK (disk objects) spread across several datastores for performance reasons. Hence the screen will show any Virtual Machines thhat are only paritally replicated in a specific sub-folder allowing for easy identification and correction

MVIV Shot 5

The fourth tab Virtual Machines shows all the virtual machines on a particular Site and of those virtual machines which are protected. For the protected Virtual Machines, the Administrator can drill down and examine elements such as whether any RDMs are configured and whether they are replicated. The drill-down will also show whether there is a Shadow Virtual Machine created at the remote site. This can be used to infer that the Virtual Machine in question is protected via an SRM protection Group at the remote site

This same tab shows the Shadow Virtual Machines that exist for Virtual Machines from the remote site that are protected by SRM. If a Virtual Machine is Failed over from London to Palo Alto, this screen will show a ghost virtual machine on the Palo Alto side indicating that the Virtual Machine has been failed over from its Production Site

MVIV Shot 6

The final tab Data Store Groups gives a final analysis of whether a LUN is valid for MVIV failover - for this to happen a number of elements need to come together such as correctly configured Consistency Groups, Virtual Machines etc.

Only if all the criteria are met will a LUN be judged eligible for MVIV failover. So in this one screen an Administrator can quickly see if a Mirrored LUN and all the structures such as DataStores and Virtual Machines built on it are in compliance by site

MVIV Shot 7

Once all the LUNs and other components are correctly configured the Administrator is offered the option of failing over (if they should choose not to use the SRM failback capability) or to failback a configuration from its Recovery Site to the Production Site - something that VMware SRM at the moment only supports manually.

Now the reality remains that the VMware SRM team has not exposed an API for us to utilise for these types of application so the MVIV application cannot recreate Protection Groups - so prior to failback, certain manual steps such as removing the old and now defunct Protection Groups may be necessary.

However what the MVIV application will do, upon Failover, if you ask it to (in the comprehensive options menu) - is remove the now defunct Virtual Machine entity on the Protected Site (enabling seamless failback) and upon fail back to the Production Site, MVIV will remove the now defunct Virtual Machine on the Recovery Site allowing you to configure the new Protection Group on the Protection Site without getting errors.

The failover and failback is seamlessly achieved as the infromation relating to Resource Pools, Folders, Networks etc. configured in SRM is populated down to the MVIV application

In this way SRM can be used to failover a Protected Site configuration to the Recovery Site and MVIV, in a fashion similar to the Celerra Failback tool, can be used to fail that configuration back to the Production Site

MVIV Shot 8

All in all the MVIV is immensely powerful in terms of the information it offers to CLARiiON and VMware administrators to help them properly configure a VMware and CLARiiON environment to successfully levereage SRM.

Many of the more important concepts that underpin successful Disaster Recovery such as identified site entities is built into the application and with it Administrators can quickly analyze their current environment to understand that it will function as expected and that the various elements, LUNs, DataStores and Virtual Machines are properly configured

The screenshots of the application to match the place holder references are included in the zip file attached

Business Continuity Concerns - driving uptake of Virtualization technologies

2009-08-31T23:37:27Z

Business continuity is a term undergoing a transformation. Traditionally associated with local and remote backup and recovery of data critical to an organisation, in the virtualized world it has becomes a much broader term. Virtualization is changing the understanding of the term Business Continuity while at the same time providing the means to extend its benefits to the entire IT infrastructure of an organisation.

Consolidation was seen as one of the key, if not the key reason / benefit of virtualization in the first wave of technology deployment. The cost benefit of extending Business Continuity across the organisation in terms of minimising downtime, achieving regulatory compliance and surviving catastrophic disasters will characterise the next wave

Today Business Continuity, at least in my experience of conducting road shows on the subject in conjunction with VMware up and down the UK, means a great deal more than Backup and Recovery. Today when I hear business Continuity mentioned it is often used as a blanket or umbrella term for three distinct areas, Uptime, Backup and Recovery and Disaster Recovery. Each of the three areas is important in their own right and there is some commonality. However, while the three concepts have some overlap, they should not be seen as substitutes for each other

Uptime is a concept familiar to those system administrators lucky enough to be able to justify the costs, management complexity and autonomy of clustering. The same awareness is true of those application administrators that have been able to take advantage of advances in the design of their applications with innovations such as database mirroring and log shipping. Uptime is essentially about preserving a service in an operational state irrespective of hardware and often operating system or application malfunction or failure at a local site or even across geographic separation. As such, uptime, is reserved typically for a small portion of any organisation's infrastructure in the physical world.

With the advent of an enterprise class hypervisor combined with a robust and capable management platform, this previous limitation on Uptime coverage has been eliminated. Technologies embedded in the Virtualization stack such as High Availability, Dynamic Resource Scheduler, Update Manager and now Fault Tolerance provide an organisation with the ability to extend measureable Uptime to their entire infrastructure as the cost benefit analysis of consolidation sees increasing portions of their estate virtualized. With a Virtual Infrastructure in place, organisations can begin providing the hallowed five 9s standard to increasing proportions of their business just in time to compete in a global wired economy that no longer tolerates or understands outages.

Backup and Recovery is what it has always been on one level, the preservation of point in time copies of vital data either captured without awareness of an application or in conjunction with the various frameworks offered by the major vendors such as Microsoft or Oracle. The comprehensive nature of any organisations recovery capability is determined by the importance of the data, the impact on critical production performance by the backup mechanism, preserving established Service Level Agreements and any internal or external compliance requirements.

On a different level, backup has also be seen as a Disaster Recovery tool with backup tapes sent off site and increasingly Virtual Tape Libraries replicated to remote sites. In all of these areas Virtualization is bringing transformation. Backup is traditionally a bandwidth intensive operation requiring dedicated networks and hosts not to mention ever faster backup media. Virtualization has transformed and violated this dynamic through consolidation. While compute and memory resources on ESX hosts are now never idle, consolidation has lead to a constraint on network and fabric bandwidth. A constraint that is unfriendly to traditional backup techniques.

Techniques such as VMware Consolidated Backup and utilising snap shot and cloning techniques possible with common storage platforms has partially alleviated these issues, but as with redefining Uptime, Virtualization is forcing an abandonment of traditional backup and recovery techniques. Deduplication, as a technology, is at forefront of the new approach to Backup and Recovery imposed by Virtualization.

With Deduplication, the huge amounts of common data created by VMFS file systems full of duplicate system partitions, application binary partitions and file server stores are recognised. Rather than backing up terabytes of essentially the same data day after day, deduplication identifies common elements and once those are established only backs up the change data. This reduction in the amount of bandwidth and resource required to backup an infrastructure frees up a Virtualized environment to focus on its core task - hosting business critical services.

Reducing the strain on the infrastructure allows greater consolidation ratios to be achieved, allows technologies such as Dynamic Power Management to be deployed for the first time and provides substantial cost savings in terms of the backup media now required. Deduplication offers further benefits in Remote Office Branch Office scenarios where the small amounts of deduplicated data, encrypted and compressed, sent back to the main sites ensures that WAN links do not need to be upgraded to achieve branch office consolidation objectives.

Deduplication technologies can be inserted into the Backup and Recovery process at any point between the client and the backup media. However, the greatest deduplication ratio and the least overall strain on a Virtualized infrastructure are seen where an agent is deployed in the VMware Guest Operating System.

Disaster Recovery, another preserve of the very limited elements within most organisations' physical IT infrastructures, is also being transformed by Virtualization. As alluded to earlier with increasing internal and external pressure to maintain services, few organisations operating on a local, regional or even global scale can afford a sustained outage let alone a catastrophic site failure.

To this end the release of VMware's vCenter Site Recovery Manager (or SRM) has provided the means for organisations' for the first time to consider genuinely protecting large portions, if not all, of their critical operations. Leveraging storage replication, SRM provides a means of duplicating a production environment at a remote site and leaving it there inactive, but up to date (in line with the replication technology being utilised) awaiting activation in the event of a disaster. In addition to providing a facility to recover an environment, in line with increasing internal and external regulatory scrutiny Site Recovery Manager allows organisations to test their recovery strategies in a way that is non-disruptive to the production services. This testing is critical to ensure that the infrastructure can be brought online in the correct order of services at the remote site. A further benefit is that the remote site need not stand idle but can host production or test and development services in its own right, selected elements of these services can be shut down to free up capacity to run production services from the main site in the event of a failover

As Organisations begin completing the first wave of Virtualization deployment, many are looking for a new rational to help build the business case for continued investment in Virtualization. A number of the benefits of Business Continuity as enabled by Virtualization as identified here are worthy of consideration, irrespective of an organisation's size.

I enclose several White Papers, Solutions Guides and Reference Architectures that may prove of interest - enjoy

Selecting the correct physical platform to support Virtualization as an Infrastructure Tier

2009-08-29T23:07:15Z

As we increasingly see Virtualization spread across many organisations' IT infrastructures, it begins to assume the form of an Infrastructure tier in its own right. This has significant implications for future purchases of the hardware and software to provision this new tier.

Virtualization has traditionally occupied a specific role in an organisation's IT offering, as a test and development environment, as a consolidation exercise for Tier 2/3 services or increasingly as a viable candidate for desktop services. With the transformation from a select segment of the overall infrastructure into one of the key tiers spanning that Infrastructure, Virtualization is forcing organisations to look at a different cost and requirements model.

Most organisations have traditionally purchased the hardware and software for their Virtualized environment following a familiar pattern of particular class of server, networking and storage and then overlaying the hypervisor and applications services. While this approach provided known costs and capabilities when Virtualization was confined to specific segment of the IT infrastructure, it is increasingly seen as an anachronistic approach, unable to meet the changing demands of the IT infrastructure as the variety and types of services migrating onto a Virtualized Infrastructure increases

This changing approach has seen new trends such as the emergence of multiple ESX clusters composed of hosts with different specifications to take account of the varying workloads, Tier 2/3 vs. Tier 1 dedicated applications. Along with varying choices of server platforms, the introduction of 10Gbs iSCSI and IP solutions has added additional choices to networking, in a similar fashion to the introduction of 8Gbs fibre channel.

Organisations can now host their Virtual Machines on low cost, low speed, large capacity SATA disks, higher speed, higher reliability fibre channel drives and ultimately for demanding applications Enterprise Flash Drives. When connecting to these storage tiers, the choices range from simple 1Gbs IP / NFS connectivity right through to 20Gb/s Inifniband. Technologies such as Storage VMotion allow the seamless movement of Virtual Machines to reflect the performance, reliability and scalability requirements of various services.

However, for organisations looking to tier their Virtual Infrastructure in the most cost effective yet performant manner, all this choice can seem bewildering. In this regard when provisioning for a developing infrastructure the last thing an organisation wants to do is limit their current and future options whether it be with servers, connectivity or storage.

To this end vendor choice is critical. Elements such as storage are particularly sensitive to this as they are viewed as possessing a 3-5 year time window for investment and to find that a particular recently obtained solution is obsolete or a barrier to the adoption of a key new technology can prove costly both in terms of project timescales, transformation and allocating resources for an unplanned infrastructure refresh.

EMC as a vendor of storage and networking solutions has sought to address this problem through a commitment to fast product development cycles, the use of state of the art yet generic hardware components and creating storage and connectivity architectures that are fully flexible and which enable organisations to upgrade or change their configurations as their environmental requirements transform.

In considering any vendor of Virtualization solutions it is critical to ensure that the latest cost saving technologies are being adopted and the options under review now do not preclude the adoption of new and different innovations within the effective lifespan of the components. Flexibility is also key criteria. As we are seeing a broader variety of services being ported onto a virtualized infrastructure the merits of equipping all ESX hosts with Fibre Channel HBAs, or deploying SATA disks for all Virtual Machines is diminishing. A truly flexible environment should be able to offer all tiers of storage and all forms of networking technologies to ensure that Virtualised services are operating in the most cost effective manner.

Included in this post are two White papers looking at two examples of EMC technology and how these offer scaleable and flexible solutions for organisations transforming their IT infrastructures from physical to virtual tiers

The growing popularity of Virtual Desktop Infrastructures

2009-08-28T18:27:28Z

It has been suggested that I turn what was orignally a discussion question into my first blog post

EMC like many other companies that provide solutions to the IT infrastructures within companies is noticing a growing interest around the concept of virtualizing an organisation's desktop infrastrucutre.

This has lead, within the market, to terms such as VDI (or Virtual Desktop Infrastrucutre) being coined to try and encapsulate and in many ways productise this trend towards pulling the deployment, mangment and securitisation of desktop images back into the data center.

The challenge most vendors and solution providers are encountering when discussing virtual desktops as a concept within individual organisations is that many in the IT community profess a view that VDI is a relatively mature and well understood technology and as such can be purchased off the shelf and subsequently configured to offer almost endless customisation irrespective of the desktop demands of a specific organisation.

VDI in simple terms as it exists today is not a panacea or quick win solution for removing the capital and operational costs of a physical desktop infrastructure. Organisation by organisation in carefully planned and undestood scenarios, VDI can offer some specific capital and operational savings - but these will not be achieved without a commitment to in mancy cases to radically transforming an orgnaisation's approach to how it manges, deploys, updates and secures a desktop infrastructure. Much of that transformation is not about technology but about people and processes

The approach that believes VDI is a mature univesal technology is usually rapidly transformed once the organisation in question begins to deploy some form of VDI solution in earnest. At this stage it is often too late. Unlike choosing the wrong hardware platfrom for an application where you have one application administrator and his henchpeople unhappy with you, if a VDI deployment fails it could lead the IT department to receiving hate mail from the entire desktop community - which usually includes the mamagment tier, not always the best of career choices

Where EMC and I suspect many other solution provioders and vendors have seen success with VDI deployments is where the deployment begins as collaborative Proof of Concept exercises. The solutions provider in conjunction with the customer carefully profile the different desktop use cases that exist within the business, eliminating those that do not fit within the still realtively confined 'sweet spot' of VDI use cases.

Once the Proportion of the desktop universe that is realistically eligible for VDI as it exists today has been identified, then further decsions need to be made around image, creation, deployment, mangment, securitisation and updating. The customer alos needs to decide how user data is stored, manged, protected and in these times of space savings deduplicated and archived.

The customer also needs to consider access to applications and whether users need to access the same desktop image on every occassion and what sort of performance profile is required for a happy user.

On many occasions when talking to customers, particularly those in large organisations with established Anti-Virus and patching regimes, the existing processes for deploying AV scans and patches in the VDI universe can lead to massive spikes in the backend storage requirements and in many cases to serious consideration being given of technologies such as Enterprise Flash Drives to alleviate the performance overhead

Included below is a link to a VMworld discussion which was how i orignally posted these thoughts. Included with that post are a series of EMC White Papers, Reference Architectures and Solutions Guides that look at deploying VMware View technology on a number of platforms (fibre channel and iSCSI), protecting that VMware View environment with deduplication technology and deploying a VMware View infrastructure at scale in collaboration with VMware and Cisco - VCE

The original post can be found here

http://www.vmworld.com/message/7112#7112