Digital Walkabout

Amazon's new offerings leave IT pros wanting more

Amazon previewed a new Data Pipeline service and made available new EC2 instance types this week at AWS re:invent, but enterprise IT pros said there's still room for improvement with its existing offerings.

 

Specifically, attendees at the Amazon Web Services (AWS) re:Invent conference here said they're looking for live migration of Elastic Compute Cloud (EC2) instances between availability zones and regions for disaster avoidance.

 

"Right now, moving workloads around to avoid outages is pretty cumbersome and manual," said Jason Deck, senior operations engineer for Atlanta, Ga.-based LogicBlox. "I want to know if it's something they're working on."

 

Netflix CEO Reed Hastings brought up the topic of live migration within Amazon's cloud during a keynote presentation this week.  

 

"VMware does this today with vMotion and has done this for many years, so it is technically possible," Hastings said. "But it's extremely demanding to be able to do that at scale."

 

Attendees at the show also said they hope to see further service automation. This would include automated selection of instances for different workloads, which was also called for by Hastings.

 

"We're really still in the very primitive first assembly-language phase of cloud computing," Hastings said. "When you have to pick individual instance types … you know something's wrong."

 

Over the next five to ten years, Hastings expects EC2 will develop to select instances based on the type of workload being deployed and move workloads among instance types as they scale using live migration.

 

IT pros looking for more automation also envision an automated "lights out" function for EC2, so that they don't have to manually shut off their inactive EC2 instances.

 

"Why don't they have a timer to turn the lights on and off automatically?" said Matt Lipinski, architect for Reed Elsevier Technology Services, based in Miamisburg, OH. "Why not have a checkbox for shutting down instances as a group or individually outside of business hours?"

 

Other wish list items cited by IT pros at the conference include geographic expansion of Amazon's regions to more parts of Asia and Russia, more cities supporting the Direct Connect service, simplification of provisioning and operational workflows in Amazon's Virtual Private Cloud, and more reference architectures and documentation.

 

Data Pipeline service automates data analytics workflows

Amazon.com CTO Werner Vogels revealed during his keynote this week that the company is preparing a new workflow orchestration service for data analytics called Data Pipeline, which can be used to set up data movement policies among repositories of data, including Amazon Simple Storage Service (S3), Elastic Block Store, Amazon's new RedShift data warehouse, and on-premise data repositories.

 

"It doesn't come within a country mile of replacing real [Business Intelligence] job management," said Carl Brooks, analyst with 451 Research, based in Boston, Mass. "But it works for a significant portion of the AWS audience, i.e. giant Web properties and online services with lots of raw data to process."

 

This could prove useful for aggregating and storing log files in the cloud, Lipinski said.

 

EC2 Cluster High Memory and High Storage instances

The two new instance types also unveiled by Vogels focus on workloads at the ends of the spectrum, the CTO said. The Cluster High Memory instance, also known as cr1.8xlarge, will offer 240 GB of RAM and two 120 GB solid state drives (SSDs) for high performance.

 

The High Storage instance, also known as hs1.xlarge, packs in 117 GB of RAM and 24 hard drives for a total capacity of 48 TB.  The high storage instance is suited for highly distributed Elastic Map Reduce workloads, Vogels said.

Foundations for building a private cloud

Building a private cloud isn't a quick process. It starts with understanding expectations and defining the cloud computing in your environment, then building on the model you've created. Enterprise IT must include the entire organization, its processes and its technologies when constructing a private cloud.

 

Here are the first five steps you need to take to conceive, build and maintain a private cloud within your enterprise.

 

1. Decide what you want out of a cloud

Journeying to the cloud is a huge trend in IT. The problem is that the term "cloud computing" means something different to everyone. To start your journey, your organization needs to be realistic about its cloud computing goals.

 

Many organizations find themselves looking toward private clouds only after they've realized the promises of virtualization, like data center consolidation, power savings and cost savings over physical hardware. Others find themselves wanting to take virtualization to the next level, with standardization and automation as part of their IT processes.

 

But few organizations are ready to work on organizational changes, tackling the harder "people problems" that traditional IT has fostered, such as silos, duplication of services, security and management of services. These are not usually technical problems but run roughshod over organizational boundaries and long-standing political domains.

 

And there are many misconceptions about the term "cloud," including an overabundance of differing definitions. One common misconception is that private clouds are completely based in virtualization. Even though virtualization usually plays a major role in a private cloud deployment, a private cloud can also just mean a shared infrastructure. Take, for example, Google's Gmail or Microsoft's SkyDrive. Both are public cloud services that don't rely much on virtualization. Instead, massive amounts of physical hardware are in use behind the scenes.

 

The same is true of a private cloud for your organization, in which a shared service is created to replace many different duplicate services, and the use of virtualization is evaluated only as part of that service's implementation. For example, a shared file server service might replace dozens of departmental file servers, and it might be implemented with physical hardware because of the incompatibility between VMware vMotion and Microsoft Cluster Service.

 

2. Have realistic expectations of the journey -- and the cloud

You should expect that there cannot be true self-service IT within your organization. IT departments have spent years wrapping process and procedure around the act of creating and managing servers, usually with good reason. Often these processes are responsible for monitoring systems, determining sizing and dependencies, documenting system designs and responsibilities, handling licensing and more.

 

Allowing any end user to provision a server or service without approval mechanisms in place might be appropriate for certain lab or test-and-development environments, but in a production IT environment it is a quick path to chaos, sprawl and outages. However, it is reasonable to expect much of the provisioning process can be automated and standardized through the use of workflow tools and approval mechanisms, like those found as part of Embotics' V-Commander or enStratus Networks' offerings.

 

Expect the journey to the cloud to be less about technological challenges and more about personnel challenges, as processes are torn down and recreated, routine tasks automated and standardization championed. An IT department that is heavy-handed and unresponsive to users' needs may not be in the right place to start rethinking itself and its work. Similarly, an IT department that is overworked may not have enough free time to pursue cloud solutions, despite the time savings the cloud would provide.

 

It is very important that management prioritizes IT work appropriately and backs up the IT department in the face of complaints about delays in other work due to the focus on cloud computing.

 

Finally, all levels of management, including human resources, will need to support a transition to the cloud. Not only will all facets of the organization see delays as IT works to improve itself, but IT workers whose primary jobs consist of the tasks being automated might also consider themselves targets for layoffs and may actively undermine the process. Plan for personnel issues and, from the beginning, communicate to staff that they are valuable and these efforts are intended to free them to do more interesting, more productive work for the organization.

 

3. Understand enterprise workloads and services

Working toward a private cloud model is difficult when you don't understand the services on which your organization relies. Documentation is key; without it the relationships among systems are hard to decipher, service-level agreements are unknown and it's easy to make false assumptions. The needs of the people using these services should also be documented so that new cloud services can be built to meet those needs. This is especially true when centralizing duplicate services within an organization. There was a reason a department built its own infrastructure instead of using shared services; find out the reason to get their buy-in and avoid conflicts.

 

Documentation also lends itself to standardization, because a standard that does not account for all needs and system design requirements will quickly have exceptions. Performance information is also crucial to moving toward shared infrastructure and cloud-based applications. A year or more of historical performance data, as high resolution as is practical, can be very helpful for determining capacity needs and system sizing.

 

4. Get on the path to virtualization

While it isn't required that a private cloud be based on virtualization, it is the common model. Virtualization usually drives certain knowledge and behaviors within organizations. For example, most virtualization software requires centralized storage. That same centralized storage will be a building block for a private cloud, so the knowledge gained in implementing virtualization is very beneficial to private clouds.

 

Likewise, virtualization is usually quite disruptive to data center networks. At the very least, it can turn static traffic patterns into dynamic ones.

 

The move toward shared computing models and cloud-based computing continues that trend and increases the reliance on networks, which usually drives up bandwidth needs. The dialogue started among your virtualization administrators, storage administrators and network administrators as a result of planning for virtualization will become crucial as you advance into the cloud, especially when planning to serve remote offices and mobile users.

 

5. Understand that standardization and automation go hand in hand

Automation is one of the key goals organizations have when moving to a private cloud. However, automation is incredibly difficult without standardization. For example, with standards for operating systems (OSes) and server builds, you can make assumptions about locations of files, sizes of file systems and authentication mechanisms. Based on those assumptions, you can script the installation of application software and middleware such as Web servers, application servers and firewall rules.

 

Standardization can be difficult for an organization that has not practiced it. But once you take on standardization, the time savings can be enormous.

 

Consider an organization that has had no standards for OSes, OS versions or build processes. Every server is different and every operation needs special attention. Procedures for patching or installing software differ each time, and success rates waver because of the variations in each host. This usually has two consequences: An incredible amount of staff time is spent performing routine tasks on these servers, and many routine tasks, like patching security vulnerabilities, are skipped because they are too difficult and unpredictable. Standardizing on one or two OSes and automating the build and application deployment processes yields massive IT productivity gains.

 

Once you've automated much of your environment, you can deliver self-service portals and service catalogs. Though it is unlikely that your organization will ever be 100% self-service-driven, many processes can be automated with workflows; the only interactions, then, are approval processes. This allows the IT department to focus on more important issues -- how to best support and monitor an application or service, for instance. It also gives application admins and developers a consistent and repeatable platform on which to build. And it means IT operations staff can build useful, repeatable procedures for handling incidents and monitoring system alarms, instead of each server being a one-off exception. It may even open the door to automated responses to alarms.

 

These are the first five steps an enterprise needs to embark on when building a private cloud infrastructure. But these are just the beginning. IT teams need to continue on the private cloud journey by putting in place procedures, such as chargeback or showback, that hold business units accountable for IT resources provisioned and consumed. And private clouds must be carefully monitored to maintain performance and keep end users -- as well as the CIO -- satisfied with the new direction you've taken IT.

In ongoing acquisition spree, Cisco snaps up Cloupia for cloud automation

Cisco Systems has been boosting its position as a leading cloud player, with its pending acquisition of cloud-based network vendor Meraki Inc., its new OpenStack distribution channel, and support for the Citrix-backed Apache CloudStack.

Cisco's cloud spending spree continued in the days leading up to Black Friday as the vendor announced intent to acquire Cloupia, a Santa Clara, Calif.-based cloud automation software provider, for $150 million.

Cloupia's software automates converged data center infrastructure for both enterprises and service providers by controlling the deployment and configuration of physical and virtual infrastructure from a single management console. Cloupia will add greater control and orchestration capabilities to Cisco's cloud portfolio -- potentially boosting the vendor's rank as a cloud management vendor of choice, said Dave Bartoletti, senior analyst at Cambridge, Mass.-based Forrester Research Inc.

Cloud players must offer cloud automation
Cloupia technology -- which will be rolled into Cisco's Data Center Group -- will help the Cisco Unified Computing System (UCS) Manager and UCS Central to manage server, network, storage and virtualization resources outside the UCS stack, said Hilton Romanski, vice president and head of corporate business development for Cisco in a blog post. Cisco also plans to integrate Cloupia's software into its Nexus switches.

The acquisition of Cloupia will compliment Cisco's Data Center strategy by providing a single "pane-of-glass" management capability across both Cisco and partner solutions -- including FlexPod, VSPEX and Vblock -- and provide open application programming interface (APIs) for integration with a broad community of developers and partners, Romanski said.

"Cisco is trying to put value-added software layers on top of its hardware to make their cloud offerings appealing, and it’s absolutely necessary to stay competitive," said Colin McNamara, director of data center practice for Nexus IS, a Valencia, Calif.-based managed services provider and value-added reseller of Cisco products.

Cisco has networking and server products, but not its own storage arrays, unlike Dell and HP, its competitors in the converged infrastructure space, McNamara said. "Cisco needs to be able to configure assets that they don't make [like its EMC and NetApp storage components]. That’s where Cloupia comes in for better management for [Cisco] customers, and it will be a powerful offering to customers."

"With the addition of Cloupia's technology, Cisco will have a better strategy for provisioning complex applications in the cloud," said Tom Nolle, president of CIMI Corp.

By acquiring a third-party management software company, Cisco is also acquiring its built-in integration efforts with other virtualization, storage and networking platforms, Forrester's Bartoletti said. He said other companies also recognize, noting that these benefits are not lost on Cisco's competitors.

Dell recently acquired infrastructure automation company Gale Technologies, and VMware acquired automation vendor DynamicOps for resource management on Hyper-V- and Xen-based hypervisors, and Amazon EC2 earlier this year.

"Vendors are buying heterogeneity -- buying the capabilities to manage more than just their own gear," Bartoletti said. "Cisco realized their converged infrastructure tools and networking equipment might be a significant part of a cloud solution, but it's not going to be the whole solution."

"Cisco knew they needed to either buy or build integrations with other platforms, and this acquisition will help [Cisco] get their foot in the door with cloud automation and management," he said.

Will Cisco build its own alternative to OpenStack?
Cisco may choose to integrate Cloupia technology into its OpenStack distribution channel to manage its Quantum API technology for programmability of both virtual and physical networks, giving Cisco's version of OpenStack differentiation, CIMI Corp's Nolle said.

As a second option, Cisco could also use its Cloupia technology to build another proprietary cloud stack, he said.

"Cisco can present the same APIs as OpenStack does along with its new Cloupia management tools, and the resulting cloud stack would look the same to the user as OpenStack, but instead would be Cisco's stack alone," Nolle said, noting that a Cisco cloud stack could be more profitable for the vendor.

"Cisco has the possibility to turn the cloud market on its ear. It's a question of how aggressively Cisco will develop the acquisition, and whether Cloupia will be used simply in a DevOps environment or to develop a future cloud platform."

Amazon RedShift data warehouse sparks buzz among IT pros

Amazon's new RedShift data warehouse service may leave traditional offerings in the dust with competitive pricing, provided enterprises can overcome data migration challenges, experts said.

The company unveiled Amazon RedShift during a keynote presentation at its inaugural re:Invent conference in Las Vegas this week, and the new offering generated excitement in the halls at the show.

"We're hoping it'll be faster performance-wise," said Ivan Jurado, chief technology architect and general manager for marketing analytics company M-Sights, which moved its entire infrastructure to Amazon Web Services (AWS) just a few months ago.

So far, the company, which has about five terabytes of data to analyze at any given time, has been running queries in AWS-hosted instances of SQL Server, but this environment is quickly eating up memory, Jurado said. "Right now we're already at 37 gigabytes of memory -- we can't get much bigger."

However, enterprises with large existing data warehouses on-site won't be able to spin  up RedShift as easily, according to consultants at the show.

"It will be good for smaller guys and those who can start from scratch, maybe, but if you have a large investment in a data warehouse, you're not going to move it all to Amazon -- you already have sunk costs into that infrastructure," said Giedrius Praspaliauskas, senior systems architect for a consulting company based on the West Coast.

Some enterprises might go for RedShift depending on where they are in their procurement cycle; if they're coming up to the end of an existing deployment's life, it's more likely they'll go for cloud-based data warehousing, one analyst said.

"You also have to look at the skills of your workforce,"  said Tony Witherspoon, senior solutions consultant for a consulting company on the East Coast. "I definitely wouldn't start with data warehousing as an entry point to the cloud -- you have to pick the low-hanging fruit first."

AWS also dropped the price of Amazon Simple Storage Service (S3) about 25% across the board -- a move that may help boost RedShift, some observers said.

RedShift specs
RedShift is available as a preview from AWS and comes in two flavors: one comprised of 16 terabyte (TB) nodes and one with 2 TB nodes. Amazon claims the platform can scale automatically to petabytes in size and will compete with IT's "old guard" vendors on price.

The 2 TB nodes are available starting at 85 cents per hour, while two 16 TB nodes with 128 GB of RAM each can run for $3.65 per hour, adding up to $32,000 per year according to a keynote presentation Wednesday morning by Andrew Jassy, senior vice president of AWS.

Typically, enterprises pay between $19,000 and $25,000 per terabyte of data per year with traditional data warehouses, Jassy said, citing statistics gathered by analyst firm ITG in June 2011. Amazon's data warehouse can run for as little as $1000 per terabyte per year, Jassy said. Amazon also claims RedShift can offer ten times the performance of traditional data warehouses, citing internal tests.

"They haven't released the methodology for the benchmarks, so it's hard to make a comparison a DBA is going to take very seriously," said Carl Brooks, analyst with 451 Research based in Boston. "But the claimed cost delta is so big that it will be attractive anyway. This is the exact same value prop as EC2."

 

AWS takes aim at 'old guard' in IT, makes move for enterprise

Amazon executives left no doubt about the company's enterprise ambitions at the AWS re:Invent show in Las Vegas this week, but few enterprises are ready to jump wholesale into the public cloud just yet.

Amazon execs pooh-poohed the idea of private cloud during the keynote, and gave a brief nod to hybrid clouds. But during a panel discussion Wednesday afternoon, enterprise IT pros said they still prefer a methodical, hybrid approach to migration into the public cloud.

For example, at McGraw Hill, out of 300 enterprise applications, a subset of 70 has been earmarked for public cloud, and the company has taken a carefully planned approach to migrating applications into Amazon Web Services (AWS).

McGraw Hill started with an "easy win," a non-mission-critical app, according to senior director of strategic initiatives Erica Eatmon, and then moved to a more complex business-critical application, the McGraw Hill Connect Suite.

"The key thing was to ensure the stakeholders were all in alignment," she said. "There were fears that moving to the cloud would impact the customer experience. The platform carries more than 2 million students -- that was at stake."

The app was moved successfully into AWS without those fears being realized, but it's still a hybrid deployment; an Oracle RAC environment supporting the Connect Suite still runs in a separate Datapipe data center.

While a collaboration and file-sharing app has been moved to AWS, access controls have to remain in-house, according to Oliver Alvarez, lead enterprise security architect for the World Bank's International Finance Corporation. "We need to maintain control and custodianship of information," he said.

Both panelists also said the migration to cloud has had its rewards. McGraw Hill's Connect Suite grew at 40% a year, and buying all the hardware to support that growth was not cost-effective, according to Eatmon.

The International Finance Corporation found a way to create a Dropbox-like environment for collaboration with over 140 different countries while still maintaining appropriate security and compliance controls, Alvarez said.

Elsewhere at the show, consultants who work with enterprise clients said it's only a matter of time before migration to the cloud takes place -- but that time is still needed for enterprise clients to get comfortable.

"People are scared and don't understand it," said Del Wood, President of l33t Logic, a consulting firm located in Waynesboro, Va. "But cost is going to force it -- businesses won't be able to be competitive without moving to the cloud."

In a way, Amazon's lead in the market hurts them, Wood said. "If there were a close competitor, there would be more perceived safety; people could split their eggs between two baskets."

Amazon's aggressive pitch to the enterprise
While citing its AWS Direct Connect and Amazon Virtual Private Cloud offerings as an accommodation to hybrid cloud deployments, Amazon's keynote presentation was strongly focused on the benefits of a purely public cloud approach to computing for enterprises.

The presentation, led by Andrew Jassy, AWS senior vice president, was especially withering toward the enterprise private cloud concept. "Beware the cloud-washers," Jassy said. "Those falling for 'private' cloud are not achieving their objectives."

Amazon's model of low margins and high volumes simply doesn't appeal to the "old guard" of IT vendors, and so they're attempting to attach the 'cloud' label to the same high-margin enterprise hardware and software products they've always offered, Jassy said.

 

 

SDN holds the key to Cloud automation

As organizations graduate from only experimenting in the cloud to entrusting it with more core apps and services, cloud providers need to reconcile the complexities that come with managing a more dynamic and massively scalable service delivery model.

On-demand cloud services are built on complicated infrastructures comprising both physical and virtual resources that may span multiple clouds. These environments need to be configured precisely to support efficient and reliable service delivery. Cloud automation, specifically in network and data center configuration and management, is critical to providing optimal service delivery in a highly elastic model. And software-defined networking (SDN) may very well be the glue that repairs all of these challenges.

The case for cloud automation
Cloud providers need tools to automate everything from configuring servers and provisioning new connections to moving workloads between and among clouds based on changing capacity requirements. Removing the manual interventions in these processes will improve efficiency and reduce providers' operating costs.

Processes that are good candidates for cloud automation are those that involve fairly standard elements that can be mapped out in a workflow. These include processes like network configuration and change management; server lifecycle management; application deployment; middleware and database management; router and switch provisioning and management; and some elements of security that involve tasks that can be repeated in other instances. 

Cloud providers can develop their own scripting tools to automate the tasks associated with a given workflow and alleviate the manual steps associated with completing a process. There are also commercial tools that automate networking services like DNS, DHCP and IP address management in addition to standard network management processes like VLAN provisioning. Additionally, cloud can use these tools to isolate network management issues and automate the remediation process.

By eliminating the manual components of routine functions like password management or the distribution of a standard virtual machine (VM) image, a cloud provider can accelerate service delivery, reduce manual support costs and limit the risk of human error. Cloud automation tools also enable providers to better manage complex, virtualized environments by applying standard, predefined and pretested processes.

There are other emerging techniques and architectures that help automate higher layer and more dynamic functions, such as the movement of workloads between and among clouds. SDN is one of the most prominent examples of an approach that supports automation in network and data center provisioning -- and it's an approach that could be optimized for a cloud environment.

Using SDN to automate cloud provisioning and management
SDN architecture extracts the control plane from the physical network topology to create an environment where switches, routers and other networking devices take their forwarding guidance from an all-knowing, centralized controller -- rather than each device individually making those decisions on its own. Because an SDN controller monitors capacity and resource requirements, it can be programmed to move workloads between resources based on capacity.

Through technologies like OpenFlow, SDN may likely become a big enabler of cloud automation. Cloud providers could use SDN to automate the cloud provisioning process so that resources, including bandwidth, could be turned up and down immediately, as required. For example, an OpenFlow-based load balancer could automatically move traffic among VMs or even between disparate clouds, depending on the "available capacity" data fed in real-time to its library.

By design, these controller-based applications are highly adaptable, scalable and more efficient than conventional appliances that rely on more static functions to complete processes. SDN might be a good fit for both intra- and inter-cloud workload transmissions. So, in addition to being useful for automating the provisioning and management of resources within a cloud, cloud brokers could also employ SDN-based technology to move traffic between clouds, based on resource availability.

Having a standardized way to automate functions like the movement of cloud-based workloads in a multivendor or multiprovider environment has an undeniable appeal. Although SDN is still maturing, it has the potential to act as an interface between resources, and several IT vendors are introducing or working on products based on OpenFlow and network virtualization. VMware has also thrown its considerable weight behind the technology with its July 2012 acquisition of SDN startup Nicira. And service providers such as NTT Communications are rolling out cloud-based services that use OpenFlow.

Whichever techniques a provider uses to industrialize its processes, cloud automation is crucial to supporting the kind of orchestration that is essential to optimize cloud service delivery. This orchestration, which unites multiple automated processes in a seamless manner, is the cornerstone of an effective cloud service.

New vision of cloud calls for new cloud architecture design

Cloud computing offers network operators an opportunity to gain new sources of revenue, but it's highly doubtful that simply hosting legacy applications on Infrastructure as a Service clouds will generate both cost advantages for users as well as revenues and profits for providers. To be successful, the cloud must have a broader base of benefits that meet everyone's expectations.

Distributed SOA: The new architecture
This new vision of cloud should represent a complete and modern IT architecture -- that is, one that displaces the application architectures of the past. This new cloud architecture design could be called "distributed software-oriented architecture " because it would combine the current notions of SOA and application componentization with cloud hosting and distributed intelligence.

With distributed-SOA clouds, enterprises could host components of applications in the cloud, use the cloud to add instances of software components to improve performance in periods of critical load, and even fail over to the cloud if internal IT elements fail. Distributed-SOA clouds could also host service features, creating a modern vision of intelligent networking that enables network and cloud operators to be players in this era of smartphones and tablets.

Expand role of SDN for cloud
The second requirement for this cloud architecture design of the future relates to networking; the cloud's relationship with the network must be integration rather than overlay. Software-defined networking (SDN) is the initiative that best represents this vision in the current market, but SDN has to be expanded conceptually to embrace every application programming interface (API) the network exposes to higher software layers. SDN also has to expand beyond the data center to control the cloud experience overall -- something that's not been a high priority in current SDN activity.

How networks mesh with the cloud is perhaps the most critical issue in cloud computing because the efficiency of that connection determines just how distributed cloud resources can really be and how well they can serve various applications' needs.

Federation is non-negotiable
The future cloud must be federated by design. Common carriers successfully sustained integrated multi-operator communications services for decades based on the principles of the public switched telephone network (PSTN) and time-division multiplexing (TDM). These services, however, focus on connectivity. They were not designed for services that integrate smart computer-based components to deliver experiences.

Standards groups have struggled with federation issues for years with minimal success, and this can be attributed to the fact that their initiatives have focused on constructing mechanisms to share functionality, rather than on mechanisms to build shared functionality. The true cloud architecture design of the future must be plug-and-play for applications and resources at all levels, with full security and accountability.

While all of these issues are recognized in the cloud marketplace, there is still no single point where they're being addressed. At this point in cloud conceptual development, it may not be possible to create such a single point, and so it will be critical for cloud providers to monitor and influence independent activities in these areas to ensure that the cloud concept that evolves is compatible with the optimum vision of cloud value for the future.

Big data, cloud evolve with the help of 'pink unicorns'

As cloud computing evolves, one particular use for it is beginning to stand out: big data. What exactly big data means and how it fits into the cloud conversation, however, are questions not easily answered.

Medio Systems Inc., founded in 2004 as an application service provider (ASP), has recently reinvented itself as a cloud-hosted provider of big data analytics, with a focus on mobile platforms. The company is also marketing its inGenius Software as a Service, traditionally aimed at enterprise customers such as T-Mobile, Verizon, Disney and CBS, at small- and medium-size businesses (SMBs).

Brian Lent, co-founder and chief technology officer of Medio, and Ivan Sucharski, Medio's data strategist, spoke about big data, the evolution of cloud computing and how the two trends affect each other.

How would you define big data for people who aren't all that familiar?

Brian Lent: I would define it as data on [such] a scale that you can't have a single department effectively manage it. Once you turn data into an operational process where you say, 'The data is going to drive our commerce engine and our recommendations and our churn modeling and our financial forecast,' now it becomes a different kind of asset, where you need to have the analytics passage sitting next to data, collocated because of the volume and the transfer. Once data is seen as a profit center, and hence there's a utility, then I think it moves more into the big data realm.

What actually distinguishes big data and cloud computing from what came before? Or are they just the same things by different names?

Lent: As an ASP, we would still license software with a very traditional approach, where you might do a license agreement for three years. Now what we're doing with the cloud-based approach is a license fee per month. It's much more measured on volume, based on monthly active users. On the technical front, I think there are a lot of differences, but one would be us making available a lot of our services through REST-based [application programming interfaces] APIs.

On the big data side, the fundamental difference is volume-based. So, when some of our customers get into petabytes, that certainly puts you in that big data camp. The other aspect of big data is the velocity at which data is changing. One of our newer customers is Rovio, the makers of Angry Birds, which has just reached a billion downloads. We're seeing more than 1.4 billion logging events in our cloud per day. They could launch a new app that could quadruple the traffic instantly.

How does big data fit into a cloud computing discussion, and vice versa?

Lent: The combination of cloud computing and big data is going to become more practical, simply because of the efficiencies of scale. The ability for any one company to keep up from an IT perspective, I think, is difficult.

With big data typically come folks that know how to work on that data, and that may be the biggest gap -- you can't just hire this talent. A friend that used to be at Google talked about data scientists as 'pink unicorns,' and said that there's only about 120 pink unicorns in the world -- true data scientists that know how to manipulate and work with big data, delivering business value. So if you think about that as a commodity and a limited resource, the question becomes, 'How do you centralize that into a cloud-based environment so everyone can get the value, but you don't have to have that person on-premises?'

Ivan Sucharski: There's elasticity, too. When you're running analytical models, you want them to run as fast as possible, but you don't need to run them every 10 minutes. So you need as many machines as you can get for the next half hour. And then they're idle for 23 and a half hours, until the next computing cycle. In a cloud situation, you've got the flexibility of that elasticity without the cost of being offline 95% of the time.

What is the most misunderstood thing about big data right now?

Lent: The notion that it's just about the size of data. The reality is, big data as a term is morphing to describe the complexities of data, and also how data is used in the enterprise. I think there's a connotation with big data that you're going to find operational uses for that data versus just storing it.

How do you see big data and the cloud evolving in the future?

Lent: You'll see the chief financial officer get engaged more into these types of decisions where they haven't to date, and get more engaged with the chief marketing officer (CMO) and the chief information officer (CIO). I think you'll see the cloud decision making moved to the CMO from the CIO. So rather than it being an IT artifact, big data and the cloud will be part of the core decision making when they go to roll out a new product.

Sucharski: Many organizations are just at the baby steps of collecting the appropriate data. Turning data into a commodity will create a new generation of individuals who are interested in what I care about, which is quality, which means that the overall quality of information will jump. Today you're definitely mining. You're digging through a lot of garbage to glean small insights.

 

As IaaS providers hit dead ends, PaaS looks better and better

IT, telecommunications and hosting service providers have considered Infrastructure as a Service as the go-to method for breaking into the cloud. But as the cloud market has evolved, IaaS has become less profitable. Platform as a Service providers may have a better chance of differentiating themselves in an already crowded market.

Domain name hosting provider GoDaddy.com recently killed Cloud Servers -- its Infrastructure as a Service (IaaS) product -- after less than one year of service, and they aren't the only provider who has made the mistake of hastily entering the cloud with IaaS offerings, said Tom Nolle, president of CIMI Corp.

PaaS providers: Avoiding cloud failure by moving up the stack

To ensure success in the cloud, providers need to know their customers. Hosting providers are not seeing success with IaaS because their customers are typically smaller companies without in-house technical support, and not large enterprises with sophisticated IT needs, Nolle said.

"If providers want to avoid a failure, like GoDaddy's IaaS cloud business, they have to climb the stack and offer higher-level services," he said. Even cloud computing giants like Amazon and IBM view Platform as a Service (PaaS) as a long-term winner over IaaS, he said.

PaaS -- the next step up the stack from IaaS -- allows providers to offer differentiated services to their customers in the form of application programing interfaces (APIs) that can enable Software as a Service (SaaS) applications. A PaaS offering serves as a virtual operating system for customers, but not every customer will want to use the same platform or operating system, which means a narrower market opportunity than a standard IaaS offering.

"Everyone selling IaaS is selling to the same buyer, but providers selling PaaS [are] selling to the buyer of their platform -- like Linux platforms," Nolle said.

Unlike with IaaS, however, the management burden for both PaaS providers and customers is lower since everyone is using the operating systems and middleware tools.

"PaaS providers can treat the users as though they are different applications in a multi-programing environment -- which can be a more efficient use of resources, and equal a greater return on investment for these providers," Nolle said.

While PaaS can help beginner cloud providers differentiate in the congested cloud market, it can still be intimidating for some service providers. Only 55% of service providers currently offer PaaS to their customers -- compared to 95% that offer IaaS, according to Sam Barnett, directing analyst for data center and cloud at Campbell, Calif.-based Infonetics Research Inc.

But hosting or service providers can easily become PaaS providers if they offer IaaS already -- as long as they are willing to invest more in their cloud business on the marketing side, and establish how the PaaS business can be structured so customers can access it easily.

"Many providers wanted to wash the word 'cloud' over their offerings -- but if they are willing to spend a little more, they can be successful in the cloud market with PaaS," Nolle said. "PaaS is higher on the value chain and displaces more buyer cost -- which can generate the potential for higher savings for customers and better profits for the providers."

Overcoming high competition, pricing obstacles

Even the larger cloud providers are struggling to keep their offerings cost-effective for their customers. Amazon has lowered the price of Amazon Elastic Cloud Compute (EC2) -- the provider's IaaS offering -- 19 times since its launch in 2006.

"Getting the pricing model right is very daunting," said Dave Bartoletti, senior analyst at Cambridge, Mass.-based Forrester Research Inc. "Sometimes it's not about the platform. It's about how [a service provider entering the cloud] can keep up with the larger players setting the price bar, and even [established cloud providers] struggle with pricing."

For a traditional hosting provider -- like GoDaddy -- pricing proved to be a race to the bottom, Bartoletti said.

"When you have established cloud providers continuously dropping pricing, providers are having a hard time keeping up and offering a profitable cloud service," he noted.

While competing with large, pure-play cloud providers is no easy task, service providers who stay ahead of the market by offering higher-level offerings -- like Amazon is doing with Amazon Web Services -- can still see success, CIMI's Nolle said.

"The beauty of going higher in the stack is [cloud providers] can start to differentiate with their own features, which makes the offering more competitive," Nolle said. "If [a provider] doesn't have a differentiated cloud at this point, it won't be a profitable cloud," he said.

 

Windows 8, SkyDrive blur the line between desktop and cloud

The new integration between Windows 8 and SkyDrive raises data security concerns and blurs the lines between the desktop and the cloud.

Windows 8 users can log in to their desktop and Microsoft's cloud services with one sign-on, through their Microsoft account. The newness of Web authentication for desktops is a potential problem for IT, and the fact that Windows 8 is linked so closely with SkyDrive definitely doesn't lessen security concerns. Microsoft accounts and SkyDrive are geared toward end users, but admins are going to have to deal with them in the workplace.

How Windows 8 SkyDrive integration works

Windows 8 SkyDrive integration begins with the end user's Microsoft account, a single sign-on Web service that combines and rebrands various other Microsoft accounts, such as Hotmail and Windows Live. End users can log on to Windows 8 computers with their Microsoft accounts, which allows for seamless integration between the desktop and Microsoft's cloud services.

Once logged in to a computer with a Microsoft account, users can easily access SkyDrive's storage and multidevice synchronization features without having to log in to any other services. Syncing desktop settings with other computers is a snap; plus, users can remotely access files on their own PCs via SkyDrive's website, an action known as fetching. The Windows Store lets users download applications or view applications already installed on their PCs.

A Microsoft account on a Windows 8 computer is treated as a local user account. By default, the account is added to the local administrators group, which means the Microsoft account has full control over the system's hardware and software. Like any local user account, the Microsoft account cannot access Active Directory resources. Such a setup means that a Windows 8 computer essentially supports two authentication models: the Web-based model that relies on the Microsoft account and the Active Directory model that permits the computer to participate in the Windows domain.

The challenges of cloud integration

This seamless integration promises to simplify desktop, cloud and multidevice experiences, making end users more productive and efficient. But IT pros are no doubt wondering what it means for the enterprise.

The Microsoft account has local administrative privileges to a Windows 8 computer, which means the account can access any files on the machine, including those in other users' directories. The account even has access to offline Active Directory accounts stored as hashes in the machine's cache. Of course, this is not new to Windows 8, but the question remains whether a Web-based account adds another level of risk, especially given the support for fetching.

Data leakage also remains a concern as companies move into the era of cloud integration. When desktops are synchronized, the settings saved to the cloud include browser favorites and histories, as well as spell-check dictionaries. Could these histories point to confidential data? Could the dictionaries include names or other data that might violate privacy and security policies?

Then there's the issue of corporate documents. With a quick drag-and-drop, a user can save just about any file to the cloud. Yet the cloud's track record when it comes to protecting against hackers is hardly perfect: In the past year, Dropbox, Yahoo and LinkedIn have all had security issues, and user account data was compromised. If Hotmail accounts are hacked, users could lose access to their SkyDrive folders and be locked out of their own computers. Plus, cybercriminals could gain access to confidential information.

Finally, if Microsoft decides that the content users have in SkyDrive folders violates the terms of the user agreement, then the company can lock users out of their accounts. Because Microsoft has the right to police all content in its data centers, the company has full access to anything stored in the data centers -- even private folders in SkyDrive.

Moving into the cloud

Undoubtedly, the cloud offers many advantages to individual users and to the enterprise at large. It's not likely we'll be turning back from the cloud anytime soon. Administrators have no choice but to face the challenges that cloud integration presents.

Yet admins must still balance these advantages against issues such as compliance, confidentiality, data leakage and document management. They must also consider what it means to support both Web and Active Directory authentication models. The industry is still catching up with Web-based authentication despite the fact that Google's Chromebook and Apple's Mac OS computers use it. The question remains whether IT has the tools it needs to safely adopt this authentication model in the enterprise.

Many questions about the Windows 8 SkyDrive integration strategy persist, and until they've been properly answered administrators should give serious consideration regarding whether they'll permit Microsoft account logins on their Windows 8 machines. If a company decides to allow Microsoft account logins, IT must make sure that the necessary policies to govern usage are in place. A lot more is at stake than Microsoft's strategy for a cloud-centric universe.