The Long Slog Forward - A Dialog Begins

Jon Toigo and I have been chatting about tactical and strategic storage decision making for many years. He posted a thread of a recent conversation on his blog site and we decided to cross post his blog.


The Long Slog Forward - A Dialog Begins

My previous post about the long slog ahead for IT business prompted a communique from my friend and fellow blogger, Mike Linett, over at Zerowait. We have been chatting back and forth via email over the past few days and I thought I would post the emails here so that others can chime in if so inclined.

ZEROWAIT:

There is a fundamental shift that is occurring in the enterprise storage and networking business. Enterprise network and storage consumers have encountered an uncertainty tax based on the unpredictable government regulatory and tax policies as well shifting models on hardware supercession and maintenance costs from hardware vendors. When combined, there is no way to accurately forecast what operational costs and support costs will be for the 24 month to 36 month time frame. Based on an inability to forecast future costs, we are seeing enterprise customers reduce their time horizons for hardware upgrade and support ROI calculations.

I feel that the “uncertainty tax” is causing enterprise network and storage consumers to look at third party support vendors. Corporate finance is asking which costs can be reduced for FY 2010? I suspect that in FY2010 organizations will try to find a rational predictive model from which to model their network operational costs. The uncertain Financial and Regulatory environment is causing corporate financial types to keep a stronger cash position, therefore CAPEX spending is rationally being curbed based on caution.

Corporations are acting like rational consumers, individuals are saving more and shopping at Walmart and Costco to cut their weekly Operating Expenses to live within their paychecks. Companies are now living within their cash flow statements, and managing risk with the security of a stronger cash position, and shopping for competitive support pricing.

DRUNKENDATA:

I agree with your view of corporate evaluations becoming more rational. But, if that is the case, how is EMC showing $1.1B in revenues per IDC from equipment sales? That is double its nearest competitor’s number for what is arguably inferior products in many cases. Even if you are an EMC fan, the long tail cost of their gear comes in the form of a warranty and maintenance contract that rivals NetApp in terms of its price.

Some might argue, as one financial analyst recently did with me, that the key to enterprise sales is really predictability. The big league consumers want an enterprise storage rig from a vendor who will still be around for three years. The CAPEX spend is either a write-off, or they lease it, which is also a write-off. All things being equal, they are taking Gartner’s cue that the most important criteria in vendor selection is “ability to fulfill.”

Everywhere else but Wall Street, I see what you are describing: close attention to economical purchasing. I have heard repeatedly that companies are dropping maintenance from the OEM and shopping it out to third parties where (1) knowledgability is there (often because the aftermarket service company is staffed by techs who were laid off by the OEM), (2) maintenance can be performed remotely (so the company doesn’t need to increase its own staff headcount), and (3) the support service provider has been around for a few years and is likely to exist over the life of the maintenance agreement.

As for equipment itself, the other-than-Wall-Street enterprises seem more interested than ever before in Red Hat, Gluster, ZFS, and other Linux/Open Source wares to provide value-add services around storage. They are not shopping for integrated one-stop-shop rigs with lots of value-add features embedded on the array controller that they may or may not use but must pay for nonetheless. Plus, the value add stuff tends to break down and requires patching, in some cases, with the same frequency as Microsoft software.

A big issue in every shop I go to is the mismatch between existing storage infrastructure and server virtualization. The latter is a huge driver of server consolidation, to be sure, but hypervisors are also gumming up I/O performance and creating real opportunities for third party I/O monitoring and measurement wares (like Virtual Instruments), for off-box I/O routing (Xsigo comes to mind), and for simply-connected “network storage” rigs – and in many cases Direct Attached Storage. Also, give the failure potential of virtual server-based hosting, I am also seeing virtualization create a lot of mindshare for data protection software. Even the hardware independent virtualization software firms seem to be garnering revenue from the decisions of CIOs to embrace VMware and Hyper-V.

What seems to be missing in all of this is a unified architectural model, complete with open management standards, that will enable cheaper storage to be integrated with virtual server environments in a predictable and cost effective way. Initially, I think that companies will use open source operating and file system features to wed storage to virtual servers. Going forward, however, I am a huge advocate of a Web Services management framework leveraging REST.

What are your thoughts? Is there an opportunity to turn a frown upside down and to get consumers thinking about the current situation as strategic, rather than a temporary response to a temporary economic downturn?

ZEROWAIT:

As a consequence of the financial panic of 2008 and 2009 there has been a tectonic shift in customers’ perceptions of proprietary features and associated values. This shift will have deleterious effects on enterprise OEM’s pricing models. For example, Enterprise customers have recognized that open source tools such as Nagios and Cacti are excellent, reliable and affordable for monitoring enterprise systems, and Apache is the choice of many enterprises for their web servers. Due to tightened budgets the value proposition of Open Source when there is a reliable support organization backing up the OS, like Red Hat, has been recognized by Enterprises.

The new model for enterprise customers is value investing in network and storage equipment. Customers will buy as they need it, and do not want to pay outrageous licensing fees based on marketing limitations. The software management costs should not trend up by terabyte, but should trend down, this can be accomplished with Open Source, but not with proprietary Hardware and Software Products. Proprietary vendors sales models are still based in the unit sales of box and license sellers, and this is a dying branch of the evolutionary storage tree. Conversely, Zerowait’s business is based on the long term service and support business, which aligns perfectly with our customers long term ROI needs. The new reality of storage is that customers are not going to buy over provisioned arrays or pay 300% markups for hard drives that the Enterprise OEM’s charge Although the major storage OEM’s and Gartner may not realize it, customers are changing their data models based on usage. Primary storage will probably stay on high priced OEM solutions for years to come, but the other 80% of data will slowly migrate to Open Source solutions that have a reliable service and support organization behind them.

Proprietary vendors claims of interoperability often require a toxic mix of patches and software upgrades to continue to work together. Over time these, patches and band aids have gotten to the point that no two networks are running the same code and revisions and they are all unique. Interoperability was supposed to leverage the idea of interchangeable parts, but now very few components can be unplugged from one network and plugged right into another without some revisions to firmware or software. Network engineering has turned back into an art more than a science as complexity has grown. Sometimes it seems that many of our network engineering customers have more in common with alchemists than with computer theorists.

If interchangeable parts made the industrial revolution possible, why is it that computer hardware has become less interchangeable over time? Perhaps the proprietary hardware vendors have wanted to lock in customers to their high priced solutions? The financial panic has broken that model though, and the marketplace is embracing open source software and POSIX very quickly due to the cost and restrictions of proprietary hardware. You can’t fight the marketplace, you must embrace its lessons.

DRUNKENDATA:

Mike, I follow you on the bulk of what you are suggesting. My concerns are perhaps further down in the weeds, or in the clouds, depending on your perception.

First, while you may be right that simpler technology based on open source may trump purchases of overbuilt Enterprise OEM rigs in the short term, both for reasons of acquisition cost and cost of ownership, I am worried about the way that companies are/aren’t considering the bigger drivers of cost.

Storage growth is predicated upon data growth. Data growth is out of control because data itself is so poorly managed. If our assessments are correct, and literally 70% of the data being written to disk is more appropriately stored in a cheap removable-media archive (or in the trash bin), then shouldn’t firms be looking at discovery and archive tools in a much bigger way?

When firms do deploy disk, it is critical that it be managed in accordance with some sort of framework. That way, you become proactive in fault resolution and develop a knowledge of trends that can help you plan your acquisitions and utilize better the investments you make. Without effective management, you again see the hardware cost curve accelerate. Yet, again, I do not see a huge push to get a hardware management paradigm in place in the companies I visit.

Finally, I don’t see companies adequately addressing the long tail cost of storage acquisition – the cost of redundancy required to safeguard data. The redundant components of infrastructure, whether in the form of a second array, a tape library, etc., required to ensure the availability of data that is being stored on the new primary rig, are not being purchased. That RAID 5 continues to exist at all is testimony to the fact that companies are not thinking through the ineffectiveness of that scheme when applied to large capacity drives. That said, mirroring is considered by many to be a cost too dear in the current economy.

In short, even with the advent of cheaper storage and open source software, I find myself wondering whether this is ultimately strategic in the absence of effective hardware and data management…

ZEROWAIT:

You have pinpointed the problem. The proprietary vendors are driven to simplify proprietary management of their hardware, while adding complexity to management of a mixed hardware environment. For hardware vendors, who are being judged by VC’s and public stock prices, units sold is a number that makes sense to their investors who are looking for short term Return on Investment. However, customers want to get a quick ROI and also a long and affordable service duty cycle out of Capital equipment. Most Enterprise storage consumers that we work with yearn to be freed from high priced proprietary licensing schemes and single vendor lock in support programs. Organizations that embrace Open Source can reduce the lock in possibilities for proprietary hardware manufacturers. Once freed from vendor lock in schemes, consumers can and will judge hardware and software by their reliability as well as their acquisition and support costs. Open source hardware and software solutions will commoditize pricing and force vendors to compete on the true value and price of their service and support. Once freed of lock in a true competitive market can flourish, which inevitably will reduce price and improve products and services.

Since there is not an open market for storage decision making, Enterprise consumers have to interpolate their changeover risk with their cost of being locked in to a particular vendor. The lock in and cost of change is well covered by the Economist Hal Varian. Our customers recognize that there can be competition within an OEM’s market for legacy support, and often the OEM’s will reduce their prices once we have a service contract in place within a customer’s data center.

In summary, the financial panic of 2008 has opened the door to a tectonic shift in the way storage is looked at by organizations. The over hyped concept of cloud storage is being considered by many, because it is an easy first baby step toward a new storage pragmatism. Everyone that has looked into the cloud model sees dangerous cells in the towering cumulonimbus of cloud storage models. Issues of security, accessibility and speed being the first concerns people voice. I predict that 2010 will be the beginning of the end of the reign of the proprietary hardware and software vendor. 2010 should also signal the beginning of storage pragmatism and organizations will rationalize their storage costs with open source hardware and storage management solutions.

(MORE TO COME)

Avatars and Symmetry

One OEM has been a big believer in the usage of Avatars for a few years now, as this story from 2007 highlights. The second link shows how the movie industry has embraced the idea, and may have created a product from it.

http://www.thehindubusinessline.com/ew/2007/06/25/stories/2007062550110800.htm
Network Appliance (NetApp) is a $2.8-billion storage networking firm up against the likes of EMC Corporation and Hewlett-Packard. Maintaining the culture of innovation amongst its engineers so products are creative and useful is a prime focus to reach up the market share ladder.
One way of developing better products is by tuning into clients’ data management woes, understanding requirements, and shaping products to reflect needs. However, the knowledge that after-sales and marketing folks get from the field does not always permeate to the engineers/developers.
And that is why the company created Mike Raja and Joe and four other personas. Joe could be a DBA (database administrator)/Chief Information Officer/any other user of NetApp’s data management products.
“This is a new approach that we are trying. Innovation is important and these personas for users of our products help continually innovate,” Louis H. Selincourt, Vice-President, SMAI & Offshore Operations, told eWorld over lunch at the company’s headquarters in Sunnyvale, California, recently.
The idea came from the book, “The Lunatics Are Running The Asylum” penned by Alan Cooper, whose company makes software more user-friendly.
“Personas create a consistent ID of our users, so we can discuss them across the company while brainstorming,” says Selincourt.

http://www.theregister.co.uk/2009/12/21/avatar_storage_effects/
Weta used NetApp kit to store the incoming data, then used a huge number of workstations and bladed servers - with 30,000 cores in total - to work on it. The NetApp filers were fitted with up to five 160GB DRAM cache accelerator cards in their controllers, the PAM (Performance Acceleration Modules) caches, to speed file access by the Weta creative people and the servers.

There is a certain balance to this when we recognize that the same OEM that has used Avatars for years has its equipment used in the production of a movie based on Avatars. At Zerowait, we still view our customers as individuals with specific needs. Probably there won't ever be a movie based on our customer service and support models.

A Milestone for Spectra Logic

I have known Nathan Thomspon of Spectra logic - www.spectralogic.com - for many years, we share a common interest in aviation. Nathan's company is celebrating its thirtieth anniversary. I had dinner with the Spectra folks recently at the Super Computer conference in Portland.

Here is a snippet on an article in his local paper on the story:

Data-storage company started in a dorm room 30 years ago
By Alicia Wallace
Daily Camera
Posted: 12/14/2009 01:00:00 AM MST

As the founder and chief executive of a high-tech company, Nathan Thompson is used to looking forward, not back.

But he's been doing a lot of the latter recently as his Spectra Logic Corp. — a Boulder- based data-storage company whose roots trace back to Thompson's dorm room at the University of Colorado — marks 30 years in business.

In 1979, Thompson was a sophomore engineering major at CU when he withdrew the $500 that was in his bank account to start a memory-board manufacturer called Western Automation. In 1987, Western Automation bought the assets of a company called Spectra Logic, and the company reincorporated as Spectra Logic in 1993 because it was a better- known name in data storage, Thompson said.

The 30-year milestone is significant, Thompson and others say, considering the dynamic nature of the storage industry.

"It is kind of amazing . . . the degree to which companies have started and died — (companies) that were competitors or the hot company around here," he said. "We've just sort of steadily marched along."

Congratulations Nathan and all of Spectra Logic.

The differences

People often ask me what differentiates the service and support that Zerowait provides from that of an OEM. I like to break it down to three component parts: Cost, Complexity and Compromises.

The cost of support services from Zerowait is typically 40% - 60% lower than the cost for support from the OEM on the same equipment. The primary reason is that the OEM typically assumes that there is no competition for their unique product and the customer will not try to use another support organization on their proprietary parts. OEM's seem to believe they have cornered the market. However, many customers have figured out that their vendors are using Off-the-Shelf components which can be (and have been) successfully cross referenced and inventoried by other support organizations. Further the OEM often recycles used equipment to service RMAs, while not exactly publicizing the fact. This leaves customers wondering what's the difference? Pretty much none, as it turns out. Therefore, a customer looking for a support organization for legacy equipment support or seeking a source for inexpensive system improvements often discovers that their OEM is more expensive for no other reason than "because I said so", coupled with an insistence on upgrading the customer to a completely new system. The problem for the OEM is that, especially in tough economic times, top-to-bottom upgrades are often not needed and cannot be justified. Cost containment is a big reason that independent parts, service and support organizations are growing.

Often OEM's will say that their proprietary systems are so complex that no independent service organization can possibly handle their equipment's unique needs. Meanwhile, these same companies habitually outsource their phone and field service to third party companies for their own cost containment reasons. As a result, complexity and simplicity are mixed in marketing messages. The reality of service and support is that OEMs design their equipment for either efficient parts replacement and repair or for OEM-service only repairs. For systems no longer under OEM contract, third party support organizations can and do provide an affordable and simple option for taking the complexity out of support and providing an alternative.

Many OEM's try to say that using third party support compromises your legacy equipment's reliability. This is perverse logic since these same companies, when asked about legacy support, demur that their engineers are no longer proficient with EOL equipment and therefore a customer needs to upgrade to maintain the highest levels of support. This logic makes it seem as if the high availability equipment that you purchased last year is now low availability simply because the OEM has superseded the equipment and no longer wants its people to know how it works. Does that make sense?

Ultimately, customers are put into a compromising situation if they do not allow competition into their data center. Without competition proprietary OEM's will not have any reason to improve their support quality or lower their support prices. That is why savvy customers are engaging third party support providers and driving support prices down and driving support quality up.

Supercomputer show

I have been traveling over the last few weeks visiting customers around the Northwest. The story is the same pretty much everywhere... Budgets are tight, and our customers are looking for value in their storage support services. The tough economy causes a lot of customers to try new and creative things to keep their costs down.

The supercomputer show in Portland was well attended and their were a lot of our customers there, I was able to catch up with many folks that I have not seen in a while.

Looking into the future

2009 has been a tumultuous business year and many of our customers have cut back on new equipment purchases. Today in the Wall Street Journal there was an article with the following headline. "Business Bankruptcy Filings Increased 7% in October"

Later in the article there was this gloomy prediction.

"On a year-to-year basis, business bankruptcies shot up 24% in October compared with the same month in 2008. Mr. Williams called that increase "substantial" and said it is a bad omen for the final months of 2009 and the first quarter of 2010.

"Bankruptcy filings are a lagging economic indicator so it's likely that we'll see bankruptcy filings increase for the next several quarters," he said.

Through the first 10 months of the year, 74,832 businesses filed for bankruptcy, a 16% increase from the same period last year.

The total number of October bankruptcies, including both personal and commercial filings, increased 20% from the same month last year. "


It is hard to be optimistic in this business environment, and yet our sales continue to grow as people embrace alternatives to save money.

Doing customer research

Over the last several weeks we have been talking to customers about what they like and dislike about their storage providers. Listening to the stories of what they are told by their OEM salesman and engineers and comparing that to the reality of service and support after product delivery reveals a startling comparison of promises versus reality.

Dealing with the big Storage OEMS's reminded one customer of The Star Trek scene of Fizzbin. I like that comparison.

Fizzbin - The Rules of the Game
Kirk: The name of the game is called... Fizzbin. Each player gets 6 cards, except for the player on the dealer's right, who gets 7.
Thug: On the right.
Kirk: Yeah. The second card is turned up, except on Tuesdays.
Thug: Tuesday.
Kirk: Ohh! Look what you've got- 2 jacks. You got a half-fizzbin already.
Thug: I need another jack?
Kirk: No. If you got another jack, why you'd have a shralk.
Thug: A shralk?
Kirk: Yes, you'd be disqualified. No, what you need now is either a king and a deuce, except at night, of course, when you'd need a queen and a four.
Thug: Except at night.
Kirk: Right. Oh, look at that, you've got another jack! How lucky you are! How wonderful for you! Now if you didn't get another jack, if you had gotten a king, why then you'd get another card except when it's dark , when you'd have to give it back.
Thug: If it were dark on Tuesday.
Kirk: Yes, but what you're after is a royal fizzbin, but the odds of getting a royal fizzbin are astron - Spock, what are the odds on getting a royal fizzbin?
Spock: I have never computed them, Captain.
Kirk: Well, they're astronomical, believe me. Now, for the last card, we'll call it a kronk. You got that?
Thug: What?

Compare Captain Kirk's exchange with the response you get when you are trying to navigate the RMA process or find out where your 4 hour parts guy is.

Repeatedly we are told that Zerowait provides easy to understand service and support, and no hassles. Things don't have to be confusing, so why do manufacturers make it that way?

A cloudy definition

A lot of the trade press is talking about cloud computing and even the Economist has gotten the bug. That surprised me because the cloud is a fairly technical concept.

"Granted, there are hundreds if not thousands of firms offering cloud services—web-based applications living in data centres, such as music sites or social networks. But Microsoft, Google and Apple play in a different league. Each has its own global network of data centres. They intend to offer not just one or two services, but whole suites of them, with services including e-mail, address books, storage, collaboration tools and business applications. They are also vying to dominate the periphery, either by developing software for smart-phones and other small devices or by making such devices themselves."

I speak to customers about cloud services on a daily basis, depending on the definition of "cloud computing" we are all using some or maybe using a lot. But for enterprise storage security reasons most of the folks we work with are leery of the amorphous cloud as their depository for critical company data. At Zerowait we don't consider remote access of data cloud computing, since most of the companies we deal with have remote data sources and applications.

If remote data centers, web enabled applications and Citrix type access are defined as part of cloud computing, then it is ubiquitous. It seems to depend on the definitions and the expectations of the users and whether they consider the cloud to be third party hosted applications and storage or not.

I think the question should revolve around who owns and runs your organization's critical databases and applications. If the lines of ownership of the hardware assets and the data are unclear you may be a cloud user already.

Wikipedia says this... "A technical definition is "a computing capability that provides an abstraction between the computing resource and its underlying technical architecture (e.g., servers, storage, networks), enabling convenient, on-demand network access to a shared pool of configurable computing resources that can be rapidly provisioned and released with minimal management effort or service provider interaction." This definition states that clouds have five essential characteristics: on-demand self-service, broad network access, resource pooling, rapid elasticity, and measured service."

Whatever your opinion is, I think everyone will agree that some stuff should not leave your organization's control.

Super Computer show in Portland

A lot of our customers and friends will be at the SC09 show in Portland this year. I have been invited to the show, and will be working with a few of our customers on how to mange and reduce their costs of storage on their 100TB- 1PB storage archives.

Over the last 20 years Zerowait has built a reputation for providing outstanding service and support to our customers. Over the last few months we have received a series of requests to help our customers with extraordinary data sets and tremendous volumes of data.

Outrageously large data sets and constrained budgets should create an atmosphere for some very interesting discussions.

Interesting article

ESJ has an interesting article that I suggest folks looking into cloud computing read. The author, Henry Newman, lays out quite a few issues with bandwidth and recovery time using a cloud storage provider.

An excerpt:

The bandwidth problem isn't limited to enterprises. In the next 12 to 24 months, most of us will have 10Gbit/sec network connections at work (see Falling 10GbE Prices Spell Doom for Fibre Channel), while at home the fastest connect available as the current backbone of the Internet is OC-768, and each of us internally is going to have a connection that is 6.5 percent of OC-768. That will be limited, of course, by our DSL and cable connections, but their performance is going to grow and use up the backbone bandwidth. This is pretty shocking when you consider how much data we create and how long it takes to move it around. I have a home Internet backup service and about 1TB of data at home. It took me about three months to get all of the data copied off site via my cable connection, which was the bottleneck. If I had a crash before the off-site copy was created, I would have lost data.

The issue that our customers are running into is in managing the backup and archiving requirements and data restoration time over the web. Henry Newman touches on these issues. Every organization values its data assets differently and needs an affordable strategy to archive data. Although cloud storage may be affordable, it may not provide the data recovery access that your organization needs to satisfy your clients.

Where is your bottleneck, and what can you do to accelerate your data recovery to provide your customers the service they need? If you can't help your customers when they call because your data is unavailable, you will be providing your competitors an opportunity to take some business away.

Software licensing

I thought this was an interesting article -

By Nicole Kobie, 5 Oct 2009 at 12:28
Confused man with his head in his hands

Microsoft knows its licensing is a problem, but don’t expect it to get any simpler anytime soon, chief executive Steve Ballmer has said.

At a Microsoft event today in London, the boisterous Ballmer was put on the spot when a question regarding the complexities of the software giant’s licensing gained enthusiastic applause from the audience of customers and journalists.

While he acknowledged the problem, Ballmer said it wasn’t likely to be solved anytime soon. “I don’t anticipate a big round of simplifying our licensing,” he said.

Ballmer admitted that Microsoft's licences had some “gotchas” in the fine print, and said that “our people” shouldn’t necessarily be hassling companies over issues in that fine print. “I’m sure we have fine print we don’t need. We’re not saints,” he said.

Licensing is a complicated issue, but it seems that vendors make it harder than it needs to be.

Unemployment & Storage Budgets

Today's news confirmed what I heard during my recent trip across the country visiting with our customers. Budgets are tight, and investment in hardware and human resources is going to be slowed by the uncertainty of what the costs of employees will be in the next couple of years.

U.S. Sept non-farm payrolls plunge 263,000

* On Friday October 2, 2009, 8:32 am EDT

WASHINGTON (Reuters) - U.S. employers cut a deeper-than-expected 263,000 jobs in September, lifting the unemployment rate to 9.8 percent, according to a government report on Friday that fueled fears the weak labor market could undermine economic recovery.

The Labor Department said the unemployment rate was the highest since June 1983 and payrolls had now dropped for 21 consecutive months.


There should be no surprise that when costs are going up enterprises need to find efficiencies. Whether it is stretching the lifespan of their legacy storage equipment to control costs, or finding creative ways to increase storage at a reduced cost with new equipment. In a global economy the most efficient and low cost suppliers will win business.

Over the years many of our customers have been asked why we don't create our own affordable hardware and software solution. And our answer to that question is that when the time is right we may. For 20 years we have been providing High Availability solutions to our customers problems. If we can bring an affordable High Availability product to market that meets our customers requirements we will.

Storage is a commodity,service is not

A friend of mine wrote me the following "Storage is the last bastion of high margins for IT vendors that make proprietary products. Ethernet, fiber channel, wireless and even operating systems have become commodity products. Storage is a commodity and should be priced accordingly." He was wondering why the storage vendors can charge so much for their products when all the components are a commodity.

William Wang the CEO of Vizio wrote " Anything that's popular will become a commodity. We're not here to build a cheap product; we're here to make the product affordable.

At Zerowait we understand why our customers and friends are perplexed by the price of commercially available storage. Everyone should recognizes that the parts are commodities, but integrating them into a High Availability solution that can withstand contact with hundreds of customer environments is tricky.

If you want a purpose built custom built solution built out of commodity parts, you can take a kit builder's approach and stitch one together fairly inexpensively. But if you want an integrated service and support solution that supplies automatic replacement parts and 24 hour technical support that will add to the cost of goods. Every company determines what risks it can afford to take in a slightly different way. Vendors of high priced storage arrays have worked hard to build their reputations for product reliability.

Zerowait's customers are looking for long term value and don't expect commodity service and support. They are looking for outstanding service and support for their high availability storage. Now in our 20th year, we provide an affordable alternative to high priced storage vendors support solutions.

Another long trip

I have been traveling around the USA visiting customers for the last two weeks. It has been a long trip and I have heard from a lot of large storage customers that budgets remain constricted. Adding storage capacity that is affordable remains a problem, as does adding staff when corporations are looking to shrink payrolls through attrition. I spoke to a few companies in California that are hiring in other states and countries, but not in California anymore. Staff shortages are creating problems for the folks responsible for data storage managment . Some are looking at outsourcing some of their storage management, but that also costs money which is unavailable.

I spoke with many business owners along the way, and their was not much enthusiasm for investing in infrastructure or people in the current business environment. Almost universally the folks I met with are looking to find incremental savings within their organizations. However, most companies are now running very lean.

It remains an interesting business environment.

Scalability, Predictability, Affordability, Reliability

My goodness the guys at Backblaze have gotten people talking about storage value. This morning I heard from one customer of ours who likes the BackBlaze price points who asked if we could provide an enterprise service and support model for a system like the one they talk about in their blog. The answer is "yes we can", but whether it makes sense to a customer depends on how they value the risk and reward for each of the following topics.

Scalability -How a customer views the issue of expandability and the related islands of storage is important to how they determine how much they are willing to spend on putting their storage in one basket. The Backblaze type solution makes sense in certain places in a Hierarchical Storage Model, but how customers value storage accessibility is a question that is very hard to answer. There are certainly customers of ours that find this price point a viable solution.

Predictability - What is the MTBF of the BackBlaze solution? There are many components in their solution that are not hot swappable. This works in their specific environment, but our customers seem to want a bit more and so the price point goes up marginally for hot swap parts.

Affordability - Our customers want a long term solution that is affordable to acquire and inexpensive to maintain. We are known for our high availability service and support and the BackBlaze type of solution could fit very easily into our business model.

Reliability - Adapting a set of Commercial Off The Shelf (COTS) components that are built for a specific solution can create a solution that meets the required reliability of the application. It is relatively easy to integrate a solution that meets a market need if it is defined very tightly.

As the leader in independent NetApp service and support, we recognize that there is interest in a solution like the BackBlaze product. But customer conversations that we have had today show that there is more interest in a storage model that has the price point of their product combined with a high availability service model.

We will be looking into this more over the next few weeks.

An affordable archive

Did you ever notice that there is an inverse relationship between data archiving budgets and data archiving growth. Data that is accessed once on a blue moon is expensive to store on spinning media, and it is hard to keep track of tapes that seem to get lost, misplaced or erased whenever there is a reporter around.

Lost backup tape prompts IT changes at NY bank - Network World (June 2008)
Officials at Bank of New York Mellon Corp. late last week said it has launched a new policy to encrypt data held on all storage devices and to limit the amount of confidential client data stored on tape drives. The policy was launched after unencrypted backup data tapes were twice lost by third party couriers this year.
*
Clients' data missing, Harvard Law warns - The Boston Globe (November 2008)
A technician lost the tape while traveling by subway between the Jamaica Plain office and the law school in Cambridge. It was one of six tapes he had put in his backpack for the trip.
*
Info on 3.9M Citigroup customers lost (June 2005)
Citigroup, the nation's biggest financial services company, said that UPS lost the tapes while shipping them to a credit bureau in Texas.
*

Missing backup tape prompts identity theft fears for JC Penney customers (Jan 2008)
GE said the tape was discovered missing last October by a worker at a warehouse run by Boston-based data-protection and storage company, Iron Mountain Inc.

Cloud based storage is also risky for secondary and tertiary storage, because there is always the possibility of a third party having a business issue, or a malicious employee.

Frequently we are asked by customers if we can help them, and the answer is "yes". When budgets are tight and mandatory compliance issues are making choices difficult, Zerowait can help with our storage options. We can help put together archiving systems of hundreds of TB's for a fraction of the cost of most hardware vendors. We have the equipment in stock and ready to configure.

If you need an affordable archiving solution give us a call. High availability storage does not have to break your budget!

Deficits, Dollars and Storage

The Wall Street Journal has an article today on the growing deficit and the possibilities of inflation that it will cause. If the Dollar's value drops I would expect that the US cost of obtaining storage components from Asia will at some point begin to rise.

This portion of the article frames the issue well:
In a new twist to an old refrain among economists, who have long worried about the effects of growing U.S. debt, they say that the huge liabilities the U.S. is taking on to dig its way out of crisis could ultimately undermine faith in the dollar.

"There has been a lot of disappointment with the way the U.S. credit crisis was handled," says Claire Dissaux, managing director of global economics and strategy for Millennium Global Investments Ltd., a London investment firm specializing in currencies. "The dollar's loss of influence is a steady and long-term trend."

On Tuesday, the Obama administration added fuel to concerns about the dollar, saying the U.S. will run a cumulative budget deficit of $9 trillion over the next 10 years, $2 trillion more than it had previously projected.

"That's going to be negative for the dollar," says Adam Boyton, a currency analyst at Deutsche Bank AG in New York. President Barack Obama also reappointed Federal Reserve Chairman Ben Bernanke, whose efforts to rescue the economy have won praise, but have also entailed pumping large amounts of freshly created dollars into the financial system.

And this part of the article shows that there still is a diversity of opinion about what will happen because of the deficits.

Mr. Buffett, for example, worries that U.S. policy makers will fail to move decisively to curtail the nation's ballooning net debt, expected by some to rise to more than 75% of annual economic output by 2013. Instead, policy makers might tolerate higher inflation, which makes existing debts more manageable but would hurt the U.S.'s creditors, including China and Japan. In this scenario, investors would demand much higher interest when lending to the U.S. government, raising its borrowing costs and making further budget deficits harder to finance at a time when an aging population will sharply boost the costs of social security and government-sponsored health care.

Investors are also growing more comfortable with the idea of emerging economies like China, Russia and Brazil playing a bigger role in shaping international finance. Some analysts, including Pimco portfolio manager Curtis Mewbourne, say emerging economies have a unique opportunity to use the crisis to reduce their reliance on the U.S. dollar., which tends to account for the lion's share of their foreign-exchange reserves.

"Investors should consider whether it makes sense to take advantage of any periods of U.S. dollar strength to diversify their currency exposure," Mr. Mewbourne wrote in a recent note.

Earlier this year, China's central-bank governor called for moving toward a "super-sovereign" reserve currency, one not belonging to any particular country. Analysts generally see such an option as unrealistic, since the U.S. wouldn't want to give up its status as the main currency in which the world's central banks hold their reserves, and any new reserve currency would require a deep and developed market where it could be traded.

There aren't yet many signs that investors are leaving the dollar. China and Japan, the biggest foreign creditors to the U.S., loaded up on longer-term Treasury debt in June, according to the latest Treasury data. China, for example, bought $26.6 billion in notes and bonds, its biggest monthly buying on record.

"The Treasury rally suggests the U.S. is facing neither an inflationary explosion nor a crisis of confidence," analysts at French bank BNP Paribas SA said in a recent note.

If the costs of US business borrowing increases because of the competition for funds, I would expect there to be an increase in belt tightening in IT budgets which eat up a lot of Capital Expenses. Many IT businesses view the US as their primary marketplace and charge a premium for goods shipped overseas to Asia, Australia and Europe. But if those markets start to grow faster than US markets for High Technology, perhaps overseas prices might fall faster than the inflation discounted dollars would justify by themselves.

Our international business continues to grow nicely, and exports might grow for us if the dollar's value declines relative to Euros and Asian currencies.

Business remains interesting.

Visiting the Northeast

Last week I visited customers in NH, MA and NY. None of the customers were talking about any green shoots in their budgets. As a matter of fact, all the customers were looking for ways in which we could help them reduce their costs of operating their storage and networks. One customer is looking into building his own ZFS storage solutions using open source software, another customer is seriously considering outsourcing their storage infrastructure and asked for our helping checking and confirming vendors claims.

It is not only customers in the Northeast that seem to be battening down the hatches on storage and networking expenses.I visited our Southeastern customers a couple of weeks ago and they are also looking at ways to save money. And while our VP of sales was visiting customers in the Northwest last week he ran into more customers that were tightening their budgets and asking for Zerowait support options and ways in which we can save them money.

Companies have figured out that they can get by without upgrading based on manufacturers obsolescence schedules. And Zerowait is growing as more companies learn that they can rely on us for high availability service and support. No matter which way you think the economy is going I think the lesson that IT departments learned is that certainly Legacy equipment can be maintained for quite a long time and save quite a lot of money.

Give us a call if you would like us review your support costs, we will be happy to help .

Perception

Many folks in the data storage industry associate Zerowait with independent NetApp service and support. But many of our customers know that we do much more than just NetApp service and support.

The picture is supposed to illustrate that Zerowait's services include Network and Storage management, security planning, installation and configuration. I admit I am not much of an artist : )

Our Professional Services team is recognized by customers around the world for our engineering excellence. Zerowait customers never sacrifice quality to reduce costs. Our team typically exceeds customers' expectations.

Our customers know they can trust Zerowait to be their high availability partner for their network and storage needs. We bring a history of high availability technical expertise and an understanding of business processes to every transaction. Our support model offers companies a quick, integrated solution for achieving the highest levels of customer satisfaction.

Storage is a big part of our business, but we do a lot more.

Moving data to the lowest cost location

Energy and power costs are a big part of the decision process when it comes to locating data storage or a complete data center for a company. An MIT PHD student has just created an algorithm to help companies serve and store data based on energy prices.

An Internet-routing algorithm that tracks electricity price fluctuations could save data-hungry companies such as Google, Microsoft, and Amazon millions of dollars each year in electricity costs. A study from researchers at MIT, Carnegie Mellon University, and the networking company Akamai suggests that such Internet businesses could reduce their energy use by as much as 40 percent by rerouting data to locations where electricity prices are lowest on a particular day.

Modern datacenters gobble up huge amounts of electricity and usage is increasing at a rapid pace. Energy consumption has accelerated as applications move from desktop computers to the Internet and as information gets transferred from ordinary computers to distributed "cloud" computing services. For the world's biggest information-technology firms, this means spending upwards of $30 million on electricity every year, by modest estimates.

Asfandyar Qureshi, a PhD student at MIT, first outlined the idea of a smart routing algorithm that would track electricity prices to reduce costs in a paper presented in October 2008. This year, Qureshi and colleagues approached researchers at Akamai to obtain the real-world routing data needed to test the idea. Akamai's distributed servers cache information on behalf of many large Web sites across the US and abroad, and process some 275 billion requests per day; while the company does not require many large datacenters itself, its traffic data provides a way to model the demand placed on large Internet companies.

As Cloud computing and Storage as a Service become ubiquitous, data centers will market themselves as low cost power providers, and high tech jobs will probably follow the equipment to locations that provide the lowest energy costs.

The Energy grid and the connectivity grid will probably begin to grow towards the lower cost providers as more organizations recognize the price advantages. As the grids mature the low cost areas will end up with a set of competitive advantages in connectivity and power over the current high cost power provider areas.

The need for legacy support

Zerowait is in the legacy service and support business. It is a niche business, that is expensive for OEM's to handle, but it is important to the end users we support. Legacy systems are inevitable because networks are never completely upgraded and often systems and networks are left as islands to be upgraded later. For example, the US Government is often left with legacy network issues as this article points out.

"HUD's problems start with its IT infrastructure. The agency has multiple grants management systems that can't share data, still renews about $7 billion in low-income grants using an inefficient paper-based system, maintains at least 16 financial management systems, and has IT systems that are, on average, 15 years old. Historically, only a small percentage of IT spending has been set aside for new development, including 2% in 2008."

Providing high availability service and support for legacy systems is a very specialized business. Many of our NetApp service and support customers can no longer get support from the OEM. The OEM has decided that these legacy customers are not worth maintaining, and their sales and engineering resources are better utilized pursuing new product opportunities. But with budgets tight and resources limited some customers are forced to maintain legacy equipment. Our job is to keep the older equipment running reliably, so when the customer can afford to upgrade he can.

In our view keeping reliable equipment running is not competing with new product sales, since the customers are not in a position to upgrade the legacy equipment. Our customers trust that Zerowait will continue to provide them with honest answers and well thought out solutions. As the Wall Street Journal editorial points out today:

"Most importantly, trust will become the critical factor. Without the luxury of time, trust will be the new currency of our times, whether in news sources, economic systems, political figures, even spiritual leaders. As change accelerates, it will remain one true constant."

Do you trust your Storage vendor to provide you with honest assessments of performance and ROI calculations?

Is NetApp going to be acquired?

Whenever NetApp acquisition rumors start I get customers and analysts calling to ask what I think of the news. Yesterday afternoon it seems another rumor was floated and folks wanted to know my thoughts.

Tech Rumor of the Day: NetApp Touted as M&A Bait
James Rogers
08/04/09 - 01:41 PM EDT

Storage may offer a bright spot for investors in the slowly-recovering economy, as M&A chatter again swirls around NetApp(NTAP Quote).

"We do believe it would make sense for a larger IT company to acquire NetApp as it would be attractive from a product portfolio and accretion perspective," wrote Brian Marshall, an analyst at Broadpoint AmTech, in a note released Tuesday. The storage specialist could be particularly attractive to a "more mature" IT company like Cisco(CSCO Quote), IBM(IBM Quote) or Hewlett-Packard(HPQ Quote), he added. Although not exactly the tech sector's equivalent of The Graduate, there has already been speculation touting NetApp as a possible target for the cougar-like IBM.

These things come in cycles and I imagine eventually something will happen which is why it concerns our several hundred NetApp support clients. Customers are looking for a safe harbor for the continuing support of their equipment, they want to make certain that their storage strategy is secure, and these rumors force them to look at other storage companies who may be able to provide a longer term solution. No one wants to have a proprietary storage solution that competition has eliminated from the path of storage evolution.

In my opinion, these rumors may help NetApp's larger storage competitors because their sales people can leverage the uncertainty about NetApp's future into sales of their products, or at least by planting the seed of doubt into their competitive sales pitch.

On the other hand, the smaller competitors like Isilon and 3Par can leverage the uncertainty about NetApp to customers who ask about whether they are big enough to compete. They can spin a story about how NetApp's future is as uncertain as theirs is, anyone can get acquired and products can get eliminated from support.

Zerowait can and will continue to provide service and support for Filers if NetApp is acquired, just as we have been supporting the NetCache line since it was sold. We will assist our several hundred customers in migrations if required, and help them with the disposition of their superseded storage assets. Our customers know that they can trust us to provide them with an honest, well thought our migration path. If there is an acquisition, and they are forced to move to a competing technology, our intention is to help our customers with a long term service and support solution that will help them migrate to a competing technology as smoothly as possible, when they need to.

Thanks for the link

In Sourcing Zerowait

Over the last few weeks I have heard from a number of companies that they are cutting back on staff. They want to know if Zerowait can assist them with their storage network management on an ongoing basis. And the answer depends on getting an understanding of their business plan, requirements and expectations.

A company that is tightening its belt has a much different set of storage requirements than does a company that is experiencing rapid expansion. If a company is tightening its belt it may be very interested in how Zerowait can help them extend the life cycle of their current network and storage environment. Our experience has shown that it is usually substantially cheaper to maintain legacy equipment than it is to replace it with newer equipment. However, often there is a bit of tinkering and tweaking required to maximize the current infrastructure. Extending the lifespan of equipment can save companies hundreds of thousands of dollars, and is a great way to free up budget dollars for a company that is still tightening its belt. Expanding companies also benefit from reviewing the abilities of their legacy equipment though. By maintaining older equipment budgets dollars can be spent on other revenue generating projects.

If your requirements are forcing your organization to look at new equipment for secondary or tertiary storage, why not look at re purposing your older storage equipment? It is still high availability and if maintained correctly will provide many more years of service. What are your requirements and where does your performance requirements bisect your cost of equipment curve? Legacy equipment has no acquisition cost, and only has a support cost. Low cost and adequate performance are often all that is required for secondary storage applications.

Finally, what are your organization's expectations of the performance requirements. Secondary and tertiary storage may not need a two hour support and service contract. Next Business Day Parts requirements may be good enough to meet the requirements and control your costs.

In sourcing Zerowait into your planning allows your organization to look at the costs of maintaining legacy equipment against the cost of purchasing new equipment. The latest equipment might have more features, but are these benefits tangible and do they outweigh the costs? TCO studies by new equipment vendors rarely show the true savings of maintaining equipment for a year or two extra before replacing it.

Why not In Source Zerowait into your network storage decision process?

What is next?

Over the last couple of months as I have watched EMC and NetApp's bidding war unfold for Data Domain it seemed to me that many folks have forgotten what got NetApp to where they are today. Dave Hitz and his team came up with a great idea for a Keep It Simple file storage system. They executed their plan well, and brought to market a niche solution that grew into a $3 Billion dollar a year company. When I met Dave in 1998, NetApp was selling the FAS630's and about to come out with the F700 series filers. The FAS630's suffered from a power supply problem, and many of them were retired after a few years, but many F740's and F760's are still running strong and reliably. The DEC Alpha processors were awesome, and those early versions of OnTap were slim and efficient.

How many other servers, routers and other network devices do you know of that are still reliably serving content to customers 10 years after they were delivered? Dave Hitz' team developed an incredibly reliable device and the market embraced it.

But NetApp diversified as it grew and now looks toward acquisitions rather than internal product development. As this article highlights:

"In the process of losing its bidding battle with EMC for de-duplication market leader Data Domain, Network Appliance (NetApp) has exposed its weakness in the de-duplication (de-dupe) market sector. It had previously developed its own technology (A-SIS) but evidently accepted that Data Domain provided a better bet. "

If I were in charge of NetApp, I would bring Dave back into the lab and let him and his team create a new reliable platform that will deliver data reliably well in to the next decade. Keep It Simple, and make it reliable, and customers will return to NetApp again and again.

If Dave can create a category killer like the F700 series again I think the future will be bright for NetApp.

Is Zerowait a Solution Provider?

Of course we are, but we don't fit the standard model. Zerowait provides creative solutions to customers' storage requirements every day. For example our Legacy NetApp service contracts provides customers who are looking to redeploy their superseded NetApp equipment into a secondary or tertiary role the support they need at an affordable price. For other customers, we provide them with the confidence they need to make certain that they have the parts and technical service required to maintain their High Availability customer requirements with older equipment. While I was in Texas last week I spoke to a customer who is still using their R150's for a critical application, they have about 30 newer filers, but the R150's are stable and running reliably. They are happy and so are their customers. Their Solution Provider is Zerowait.

There are many alternatives to purchasing new equipment available to the savvy network and storage administrator. Zerowait can provide storage additions, upgrades and support services to customers who are looking for creative solutions at an affordable price.

Our business continues to grow in this tough economic environment, which means that more customers are embracing solutions provided by Zerowait.

Another week of travel

This week I visited customers in the Southeast. It is always great to stop by and see our customers and see how they are coping with the economic environment. Most of the customers I saw are expanding their use of Zerowait's services and asking us to look at taking over additional pieces their data infrastructure.

While I was in the office of one customer this week he asked if I had time to meet with a friend of his who needs NetApp support and equipment. He called his friend up while I was in the office and made an appointment for me to visit his friend's company the next day.

Providing high availability service and support takes a dedicated team of professionals. And while I am on the road visiting with customers I hear stories about how our team has saved their bacon time and time again.

Vendor TCO estimates might not be realistic

I was not surprised to read this article on Byte and Switch today. I have long thought that performance and Benchmark numbers were suspect in the storage and networking world.




Highlights of the article:
* Driven by a general sense that benchmarking practices in the areas of file and storage systems are lacking, we conducted an extensive survey of the benchmarks that were published in relevant conference papers in recent years. We decided to evaluate the evaluators, if you will. Our May 2008 ACM Transactions on Storage article, entitled "A Nine Year Study of File System and Storage Benchmarking'", surveyed 415 file system and storage benchmarks from 106 papers that were published in four highly-regarded conferences (SOSP, OSDI, USENIX, and FAST) between 1999 and 2007.

Our suspicions were confirmed. We found that most popular benchmarks are flawed, and many research papers used poor benchmarking practices and did not provide a clear indication of the system's true performance. We evaluated benchmarks qualitatively as well as quantitatively: we conducted a set of experiments to show how some widely used benchmarks can conceal or overemphasize overheads. Finally, we provided a set of guidelines that we hope will improve future performance evaluations. An updated version of the guidelines is available.

We believe that the current state of performance evaluations has much room for improvement. This belief is supported by the evidence presented in our survey. Computer Science is still a relatively young field, and the experimental evaluations need to move further in the direction of precise science. One part of the solution is that standards clearly need to be raised and defined. This will have to be done both by reviewers putting more emphasis on a system's evaluation, and by researchers conducting experiments. Another part of the solution is that this information needs to be better disseminated to all. We hope that this article, as well as our continuing work, will help researchers and others to understand the problems that exist with file and storage system benchmarking. The final aspect of the solution to this problem is creating standardized benchmarks, or benchmarking suites, based on open discussion among file system and storage researchers.

Our article focused on benchmark results that are published in venues such as conferences and journals. Another aspect is standardized industrial benchmarks. Here, how the benchmark is run or chosen, or how the results are presented is of little interest, as these are all standardized. An interesting question, though, is how effective these benchmarks are, and how the standards shape the products that are being sold today (for better or worse).

The goal of this project is to raise awareness of issues relating to proper benchmarking practices of file and storage systems. We hope that with greater awareness, standards will be raised, and more rigorous and scientific evaluations will be performed and published. Since this article was published in May 2008, we held a workshop on storage benchmarking at UCSC, and we presented a BoF session at the 2009 7th USENIX Conference on File and Storage Technologies (FAST). We have also set up a mailing list for information on future events, as well as discussions. More information can be found on our Website, http://fsbench.filesystems.org/.

Green Shoots or Dead Roots?

Over the holiday weekend I got together with a bunch of my friends who own businesses or are executives in companies. We had a chance to talk about what they see in their businesses and there was not a lot of optimism for economic growth in the USA. Many of the companies they represent have stabilized their businesses, but they are not expecting domestic growth this year. The consensus was that a lack of Capital Expenditures is forcing companies to spend more on maintenance instead of upgrades and replacements, but even operating budgets have been curtailed.

My informal research coincides with a report I saw by Reuters today. Some points in the report are below.

* The services sector represents about 80 percent of U.S. economic activity, including businesses such as banks, airlines, hotels and restaurants.

* Both manufacturing and service sector reports show signs that the 18-month-old U.S. recession, the most protracted in decades, may soon end.

* The Conference Board, a private research organization, said its Employment Trends Index slipped to 88.4 from a downwardly revised 89.1 in May. It was originally reported at 89.9.

* "Compared with the beginning of the year, the decline in the Employment Trends Index has significantly moderated," said Gad Levanon, senior economist at the Conference Board. "We therefore expect job growth to resume around the end of the year. "However," he added, "over the last month, leading indicators of employment were mostly disappointing, suggesting the Employment Trends Index is still seeking a bottom."


Green Shoots or Dead Roots - I think it depends on what sector of the Economy and what business you are in.

Analysts project storage growth

According to an article by Chris Mellor :

The Enterprise Strategy Group is forcasting a sixfold growth in file archive capacity, from a little over 10,000PB in 2008 to 62,000PB in 2012

This seems to one of the few bright spots I have seen in the last few months when it comes to economic growth . I see a few interesting points that this growth will have on ever tightening IT budgets.

1) If budgets remain tight, customers will have to seek cost savings on legacy equipment to purchase new hardware to handle the growth. Many OEM's raise support prices on legacy equipment as an incentive to current customers to purchase new equipment. Since budgets are tight customers will be seeking more third party support to reduce their operating expenses on their legacy equipment. That should be good for Zerowait.

2) Customers will be looking for better ways to manage their RAID arrays to gain more efficiencies out of current hardware. Whether they choose compression techniques or de-duplication technologies, Storage Resource Management is still going to be a headache.

3) My recent overseas trip clearly demonstrated that customers around the world are seeking ways to squeeze more out of their storage budgets. Large international companies and smaller regional enterprises are looking at which state or country is offering the lowest cost data center.
Clearly storing and serving more data is going to take more power, and therefore there are advantages for storing data in locations that offer cheap, reliable and abundant electrical power and connectivity.

Most of the companies we deal with have three tiers of data - Primary, Secondary and Tertiary data sets. Primary data needs to be 'connectivity near' the users but as data gets less critical it can be 'connectivity farther' away.

In the future we may see more data centers in low cost locations being filled with high availability legacy equipment that is cheaper to maintain than new equipment for the secondary and tertiary data sets.

Economics will force folks to ask "why are we serving data from the most expensive locations?"

If headlines like the one below continue companies are going to have to cut costs to survive:

Unemployment flirts with an ugly truth--the possibility of hitting a new post-Depression high.

Isilon

Over the last couple of weeks I was visiting customers internationally and I was surprised at how many customers are trying out www.isilon.com . The reason they are trying Isilon is price, or at least that is what I was told. According to the customer comments I heard, NetApp seems to be pricing itself out of these high performance storage markets. Whether this is motivated by customer budget decisions or a NetApp international pricing policy I can't tell. But for whatever reason, Isilon has now got a foot in the door of the data centers I was visiting.

I was surprised not to see EMC and IBM making more headway into NetApp's international customer base, since Isilon is. In times of economic turmoil I would expect customers to go to the safety of a major brand, but if price is the major motivator of these decisions, Isilon may be buying customer loyalty. The installations are too recent to have meaningful long term reliability results.

I don't run into Isilon very often in the US and Canada, so it might be that Isilon is targeting NetApp's international customers. Or it could just be coincidence, and there is no correlation at all.

Sometimes technology markets work in mysterious ways, and the storage market is currently reinforcing its reputation.

Traveling

Over the last couple of weeks I have been traveling to visit customers in many locations. Can you guess where I was last week from the picture?

Our domestic and international customers are looking at how to squeeze more out of their IT budgets and have been asking us to provide support for other parts of their networks. We are reviewing their requests and looking at whether we can provide high availability support for more of our customers' network components.

Providing outstanding High Availability support is challenging, although we must be doing a good job as our customers are requesting us to take over support for additional hardware within their networks.

A very good article

Often the trade press is filled with regurgitated press releases which don't give a very good analysis of the companies and strategies that are winning customers and markets. But I just read a very good article that I would suggest you read.

NetApp has been losing both mind and market share to the more aggressive EMC, which has partnered with Cisco. With Oracle having acquired Sun (and with it, Sun’s storage business) and HP expected to start building its own storage systems, there are few takers for NetApp. As a result, the company’s only option was to start buying up smaller but fast-growing players such as Data Domain. For the longest time the knock on NetApp was that it wasn’t able to buy and digest companies, as best evidenced by its struggles with its Decru and Spinnaker acquisitions. Now it’s being straight-up outbid by EMC, which is offering cash. EMC, sensing blood, wants to run NetApp out of town.

Our business is growing in the tough economy because individuals and the market are recognizing that there is value in maintaining their reliable legacy equipment. The battle for Data Domain may be a harbinger of things to come. But my crystal ball on this subject remains cloudy.

But as budgets remain tight, I expect more companies to ask for lower cost service and support for their enterprise storage systems.

Growing in a shrinking market

According to many analysts the enterprise storage market has been shrinking and NetApp and EMC are slugging it out for market share.

The NAS (network attached storage) market declined 6.7 percent year over year, led by EMC with 39.0 percent revenue share and followed by NetApp with 28.7 percent share.

Coincidentally, we are seeing growth in our service and support business this year, as companies look to save on maintenance and extend the lifespan of their legacy equipment. Extending the time between upgrades is one way to get more value out of your storage infrastructure, another way is to manage your storage resources better.

Creating a plan by which you can consolidate your existing storage, and reduce service and support expenses can stretch your budget dollars. That is why more companies are coming to Zerowait and asking us to provide them a free storage analysis and quote on their current storage infrastructure support. Superseded High availability storage equipment can still provide many years of service to most organizations for a fraction of the price of an upgrade.

Courting Data Domain

Data Domain may be the most popular company in the Storage niche in the last few days. In hard economic times it pays for customers to want to save money by increasing their storage efficiency and de-duplication makes a lot of sense if a customer can see real cost savings.

As a larger company with more large storage customers, EMC has a bigger market than NetApp does within its existing customer base. And they may be able to use DD to help their customers save money by reducing their storage requirements. Also, since many DD customers are also NetApp customers it gets EMC into the door of existing NetApp accounts.

EMC seems to have a better history of integrating companies than NetApp does, and so from a customer and employee point of view folks may want to see EMC win this battle.

During a late Monday afternoon press conference, EMC CEO Joe Tucci promised that his company would run Data Domain as a product division within EMC, and would increase its R&D investment in Data Domain technology. "We look forward to giving the Data Domain employees a warm and heartfelt welcome into the EMC family. I would like to remind everyone EMC has a good track record of integrating companies, retaining and growing key talent and achieving results," Tucci said. The CEO didn't mention the less-than-stellar integration of VMware Inc. technology into EMC, though to be fair, that acquisition has been a financial success so far.

On the other hand, NetApp does not have a good track record with at least some of its acquisitions, notably the Spinnaker Corp. deal in 2003 and the acquisition of Decru Inc. in 2006. Both companies went through a very rocky integration road with NetApp. "I do happen to believe that NetApp does acquisitions miserably," said Arun Taneja, founder and president of the research firm, Taneja Group Inc.

De-duplication is another patch on the storage resource management problem. Storage is growing and as it grows it gets more complex to manage, de-duplication shrinks the footprint but adds another layer of management.

De-duplication is not the answer to the storage resource management problem.

NetApp and Data Domain

Over the last week many people have asked my opinion of this acquisition. I was not surprised that NetApp purchased Data Domain, I know quite a few former NetApp folks who work there, and it will be interesting to see how they are re integrated into the NetApp fold. I think Ken Hibbard is perhaps the most senior former NetApp employee to be rejoining the company .

From Hoovers. com Mr Hibbard comes to Data Domain from Network Appliance, where he was vice president of product engineering. He has joined Network Appliance in 1998 and was named vice president of software development in February of 2000. In January of 2001, he became vice president of product development and East Coast operations.

It looks as if the Data Domain folks understand the NetApp product development cycle and therefore may be able to integrate their products into the NetApp Line card a lot quicker than the Spinnaker products have been integrated. I certainly hope that the integration of Data Domain's customers and employees works out better for them than the Topio acquisition did for their customers and employees.

NetApp is closing down its Snap Mirror for Open Systems product, which came as the main part of its $160 million Topio acquisition.

Topio was bought in November, 2006, and its main product was an any-source-to-any-destination replication product called the Topio Data Protection Suite.

According to an article in CXO

... Data Domain lacks the customer base of large vendors and has been facing market challenges competing against Quantum.

In today's environment customers are looking for the best value they can get for their storage dollar. The question will be if NetApp can sell Data Domain products at an aggressive enough price point, after paying such a premium for the company?

Over the next few months it will be interesting to see if NetApp can integrate Data Domain offerings into their product line and execute a sales strategy that works better than their recently ended Storevault strategy did.

6:31 PM EST Tue. Feb. 03, 2009 NetApp has quietly decided to kill its line of storage appliances for SMBs, a move which caused mixed feelings in the storage vendor's solution provider community. NetApp told some users and partners that it has decided to end the availability of its S550 platform, which was the last product in its StoreVault line of appliances. NetApp unveiled its StoreVault line in May of 2006 as a way of entering the SMB market.

Impressions from the EMC World show

The EMC World show was great. From a trade show marketing point of view it was probably one of the best shows we have ever attended. The attendees understood the storage industry and were looking for answers to their storage and budget issues. We saw many of our old friends within the industry and a few of our customers at the show.

The EMC folks put together a well organized and smoothly run trade show.

EMC world day one

The first day of the show at EMC world was really great. Our trade show booth was busy the whole time, and there were a lot of folks that really were happy to see us there. We saw quite a few of our customers from business and government. There were a lot of surprised looks and comments from folks who wanted to know what Zerowait was doing at the EMC show. But once we explained why we were there it made sense. NetApp legacy equipment support makes sense to a lot of customers with mixed EMC and NetApp environments.

I was really surprised by how many of the attendees were from overseas, this is really a great show.

EMC WORLD Next week

Next week Zerowait will be exhibiting at EMC World in Orlando at booth 149. It is going to be a great trade show and we are looking forward to seeing many of our customers and friends there. Even in the tough economic environment we are experiencing the EMC show is drawing a lot of customers. This should not be that surprising since it is really one of the only targeted events for folks in the big storage business.

We hope to see you at the show.

Are Dot Hill's shipments hurting Xyratex sales

Many years ago NetApp purchased their disk arrays from a company called Eurologic. These arrays known as the NetApp FC7, FC8 and FC9 were quite reliable and Eurologic was working on a new set of arrays when Xyratex came out with a variant of their RS1600 with 14 disks for NetApp. These disk arrays known to NetApp customers as the DS14 have been quite reliable and Xyratex has been selling quite a few of them.

NetApp has been looking to cut costs though and Dot Hill came in with a product that is significantly cheaper than the DS14 variants. According to Dot Hill:.

Pursuant to our Development and OEM Supply Agreement with NetApp, we are designing and developing general purpose disk arrays for a variety of products to be sold under private label by NetApp. We began shipping products to NetApp under the agreement for general availability in the third quarter of 2007, and net revenue from NetApp increased significantly during 2008.

In this tough market NetApp is going to have to find significant ways to cut its costs on hardware sales. I have heard rumors of a significant scaling back of NetApp Sales Engineers in the field. This will be a pity, since NetApp has some of the finest field engineers in the industry. In my opinion, if they cut back on their service and support there is not much to differentiate NetApp from every other hardware company. This may cause NetApp sales to erode further over time.

If NetApp sales are down, that would mean that Xyratex sales are down, and if Dot Hill is the new low cost vendor to NetApp it would be a double hit to Xyratex gross revenue numbers. The Xyratex folks were a loyal vendor to NetApp, I hope things turn out well for them over time.

I'll bet someone is pitching a cheaper array to NetApp to take Dot Hill's place currently.

A sense of humor

Many times a day we get to laugh with our customers over something that happens or an interesting turn of a phrase in a piece of marketing material or in press release. Sometimes people send us an email with a funny turn of a phrase and today we got a good one.

"We have received a support renewal quotation from NetApp at post-warranty rates and fell off our chairs. (Have you heard this story before?) "


NetApp builds great products, and they are very reliable. But NetApp is a box selling company and not a service and support company. Therefore, NetApp needs to keep renewal support quotes high to incentivize customers to purchase new equipment. There is nothing wrong with that business model, and over the years it has worked very well for NetApp and most of their customers. But there are customers who value longevity in high availability product support, and that is where Zerowait shines. If NetApp customers are happy with their equipment and don't want to buy the latest model, we provide them with legacy service and support at a very reasonable price point.

It is a really simple value proposition for most folks to understand.... if you are happy with what the equipment you have, we will provide you support at a price that will keep you happy. Even in today's budget constrained situations we can help you stay seated on your chairs.

Affordable Long Term High Availability.

Over the last few months our business has been holding up well, we have seen an uptick in business over the same quarter last year and we have picked up quite a number of new customers. Many of the new customers have known about us for years, but when times were flush they did not need to cut back maintenance costs. But the economic winds have shifted and while many people are busy pointing fingers and assigning blame we are concentrating on helping our customers get through the rough patch. Zerowait is a service business and the longer we keep our customers happy, the better our business will be. Most of our new business is still gained by word of mouth referrals, which means that we must be doing things right by our customers.

As a service business we are concentrating on the long term, and not the unit sales going out the shipping dock's door. By using our service and support our customers can concentrate on their long term issues, because they know that we will keep their Filers running reliability for them for the foreseeable future.

Today's news...

WASHINGTON (AP) -- The economy shrank at a worse-than-expected 6.1 percent pace at the start of this year as sharp cutbacks by businesses and the biggest drop in U.S. exports in 40 years overwhelmed a rebound in consumer spending.

The Commerce Department's report, released Wednesday, dashed hopes that the recession's grip on the country loosened in the first quarter. Economists surveyed by Thomson Reuters expected a 5 percent annualized decline.

Instead, the economy ended up performing nearly as bad as it had in the final three months of last year when it logged the worst slide in a quarter-century, contracting at a 6.3 percent pace. Nervous consumers played a prominent role in that dismal showing as they ratcheted back spending in the face of rising unemployment, falling home values and shrinking nest eggs.

April 26 (Bloomberg) -- The U.S. economy will continue to contract “for some time to come,” said Lawrence Summers, director of the White House National Economic Council.

“I expect the economy will continue to decline,” with “sharp declines in employment for quite some time this year,” Summers said today on “Fox News Sunday.”