Tuesday, July 31, 2007

"To err is human. To blame it on somebody else is even more human." Arthur Bloch

Yesterday one of my readers suggested I read the Blog of Dave Hitz. Dave is a founder of NetApp and has made a lot of money with his ideas and the ability of his staff to create and market his ideas to the world's storage markets. It is because of his success that Zerowait is a vibrant, growing company servicing the legacy products that his company created but no longer wishes to maintain.

What concerned me was this comment on his blog.

"One of my frustrations with capitalism is that – on average – corporations seem much less interested in doing what’s right than individuals. (Perhaps spreadsheets and PowerPoint presentations somehow inhibit moral behavior. Topic for another blog.) "

Dave has the power to change his company's sales and marketing tactics, after all he is the founder of his company. If he is frustrated he should work to change the company from within. Dave can insist that marketing & Sales presentations are honest, he can and he should. A business's ethics start at the top and filter down through the ranks.

Long term value in any company depends on integrity as perceived by its customers, vendors and employees. Zerowait provides disaffected NetApp customers with high availability and affordable monitoring, maintenance and management for legacy NetApp equipment. Our customers recognize that we provide the tools to get the most value out of their storage infrastructure with our service and support solutions. Ask our customers, they will tell you !

Monday, July 30, 2007

It looks like the folks at Sun are going to get serious about going after NetApp's NAS position in the market. Over the last year it seems to me that NetApp is focusing on SAN installations more, which may create an opportunity for SUN.

This article hints at what might be coming out.

Murdock also refused to talk about Sun's FISHworks project – billed as a NetApp killer.

Sun has put some of its top Solaris engineers in charge of a software/hardware effort meant to create a solid network attached storage (NAS) appliance. The company demoed this project to analysts early this year, although it refuses to give reporters the same honor.

We did track down Adam Leventhal, one of the FISHworks leads and co-authors of DTrace, at OSCON. He revealed that the product should ship early next year and that it includes some special sauce above Solaris for handling storage.


It should be pretty easy for SUN to implement a cluster fail over like NetApp's Java Virtual Machine solution.


Perhaps this solution is a result of the Sun purchase of STK and the mixing of the talent pool of the storage engineers from both companies? It will be interesting to see what happens as this product comes to market.

Friday, July 27, 2007

Jeff Browning explains NetApp Snapshots and the trade off's in this blog entry very well.

It is worth reading the whole blog, his explanations are better than anything I could write.

Here are some short excerpts...

"NetApp’s snapshots actually occurred as a result of serendipity (i.e. a happy accidental discovery). The design of NetApp snapshots is as an artifact of the design of NetApp’s file system called WAFL (Write Anywhere File Layout)."

"WAFL is so named because it never overwrites existing blocks in place when updates occur. Instead it writes new blocks containing the updated data, and then frees the old blocks. Therefore, a snapshot can be assembled by simply retaining the old blocks rather than freeing them. No additional I/O is required to do this, which leads to NetApp’s accurate claim that their snapshots have no write penalty."

"What is extremely clear, though, is the choice to drink Kool-Aid – anyone’s Kool-Aid – is a fool’s choice. You need to look carefully at both types of snapshots and make an informed choice about which is appropriate for you in the context of your particular "workload. Snapshots are wonderful, and they benefit the Oracle user greatly. But they are not without risks and trade offs. To contend otherwise would be dishonest."

Recently, Jeff Browning's posts have provided a real insight into the way NetApp technology and sales departments compete for a customer's attention while serving their Kool-Aid.

Thursday, July 26, 2007

Over the last few months there has been a significant increase in the interest in Load Balancing as a solution customers are looking at to provide high availability to their internal users and external websites. Coincidentally, the folks at Barracuda Networks have brought to market a new lower priced load balancing solution than that offered by the big players in the content delivery marketplace.

From the conversations we have had with our customers it looks like there will be an up tick in the usage of Load balancing to provide reliable disaster prevention solutions for a number of our customers.

The pendulum seems to swing back and forth between network bottlenecks and storage bottlenecks every few years. It could be that we are seeing this happen again now.

Wednesday, July 25, 2007

Round Numbers are Always False - Samuel Johnson

Lately I have seen a lot of babble about Green Data Centers. Cristóbal Conde, says...
“According to some reports, the energy consumption of server systems doubled between 2000 and 2005. Companies now spend as much as 10 percent of their technology budgets on energy. "

Eliminating spam on your Storage that keeps your email is an obvious way to reduce your requirements for disk space. One way to do that is to teach your users what the Delete key does, another way is to start spam filtering. At Zerowait, we use the Barracuda Spam firewalls and we find them to be quite nice, and really affordable also. If you have not looked at a Spam firewall yet, it might be time to start thinking about what your costs are for storing all the Spam on your mail server. If you are after a reduction in spinning media on your storage system this might help. But I don't know if it will actually save you any power. It will make your Email administration easier though.

Monday, July 23, 2007

Are load balancers disruptive to NetApp software licensing fees?

One of our customers has been asking questions about whether they should take the plunge into NetApp metro clusters. And at first glance the technology looks pretty interesting. But then we remembered that Jeff Browning, the former NetApp engineering big shot, mentioned that NetApp clusters run as a Java Virtual machine, and he implied that it is not a very elegant solution. It works though. As Jeff says...
NetApp's version: In the event of failure, the OS environment representing the failed member of the cluster reboots inside a virtual machine environment on the surviving member, running under, you guessed it, Java. Then this virtual machine Java thingy takes ownership of its storage objects, and continues whatever it was doing. With lots and lots of overhead. Goody!

When we reviewed the costs of the licenses that are required and compared that to the costs of reconfiguring the network to use Load balancers and two sites with snapmirror, the costs seem to favor a load balanced scenario.

NetApp likes to talk about disruptive technologies, but in some ways an old technology like load balancing might be disruptive to their pricing model.

Can NetApp improve its cluster technology and lower its costs? Can an old technology like Load balancing and some network engineering produce higher reliability at a lower cost? Disruptive technology works in strange ways, especially when customers are driven to reduce the cost of their storage infrastructures.

Wednesday, July 18, 2007

When a person cannot deceive himself the chances are against his being able to deceive other people.- Mark Twain's Autobiography

I found this article interesting, in light of recent articles about NetApp's problems with its channel partners.


Warmenhoven declined to comment on whether orders from customers had recovered to normal levels after an abrupt decline in April led NetApp to forecast a 6 percent to 7 percent drop in revenue from the fiscal fourth quarter to the current first quarter.

NetApp stock is down about 22 percent since the May 23 announcement, which surprised investors accustomed to better-than-expected earnings reports from NetApp. Before May 23, the shares had been up almost 50 percent since August, 2005.

Warmenhoven said in the interview that NetApp traced the drop-off in orders to 22 of its largest customers, mainly big U.S.-based companies that failed to sign contracts before the quarter ended. Orders from government customers and large companies headquartered outside the United States remained strong in the fourth quarter, he added.

"It's the top accounts," Warmenhoven said. "It's across a lot of different verticals," or industries, including financial services and energy. "It doesn't make a lot of sense. I've never seen anything like this."

The company has cut back on hiring following the order decline, said Warmenhoven, adding about 200 employees in the current quarter compared with 400 in the fourth fiscal quarter. Still, Warmenhoven said he is optimistic about the NetApp's prospects.



While traveling around Silicon Valley, I noticed that NetApp seems to be still building in its complex on Java Drive. So they must not be expecting the slowdown to last long.

Saturday, July 14, 2007

The truth about NetApp partnerships leaks out... incrementally

An article in CRN seems like deje vu to Zerowait who also had deals with JPL that NetApp decided we couldn't handle once we had established the account and had already sold them plenty of equipment. The registration program has always been a joke and nothing has seemed to have changed in the unnecessarily adversarial relationship between NetApp and its VAR's.

NetApp has always treated VAR's as a missionary sales force only valuable for "incremmental sales", and once a good customer is found they take the future deals for themselves, leaving the VAR with nothing but a lot of wasted effort.

A Network Appliance solution provider working with that vendor's StoreVault S500 entry-level storage array moved quickly to take advantage of a couple potential opportunities only to cry foul when he said he was thwarted by the vendor in pursuing those opportunities.

Ron Robinson, president and CEO of Innovative Technology Data Storage, an Atlanta-based storage solution provider, has been engaged with NetApp in a long-running battle over whether the vendor and its direct sales rep, who focuses his sales on NetApp's FAS line of midrange and enterprise storage appliances, is unfairly preventing competition from Robinson selling the vendor's entry level StoreVault line.

Robinson also accuses NetApp of violating its own dealer registration policy in the case of another customer who had been purchasing from a local solution provider but who was interested in making a deal with Robinson.

In the first deal, Innovative visited long-time NetApp FAS customer Cingular, as the wireless division of AT&T is still commonly referred to, and found interest in multiple units of the StoreVault for use as low-cost remote backup appliances, Robinson said.

While Cingular was planning to purchase FAS 270 arrays direct from NetApp, Innovative registered the deal as a StoreVault deal. "But then the NetApp rep heard that we were talking to Cingular, and didn't like it," Robinson said. "So Cingular called us to stop the deal."

The reason was simple, Robinson said. "The NetApp reps had no vested interest in selling the StoreVault," he said. "If they sell FAS or another solution, they don't want the customer to see other low-cost solutions."

Ed Smith, the local NetApp FAS sales rep, told NetApp's local StoreVault sales rep that Innovative can't go into Cingular with StoreVault, Robinson said. "The local NetApp sales rep doesn't like me because I sell the S500 against them even though its all NetApp product."

This is priceless...

As for Krishnan's excuse that NetApp didn't know what department at JPL was looking to make the StoreVault deal, Robinson said he presented all the relevant information to NetApp when registering the deal.

"Every opportunity is an RFP (request for proposal) number," he said. "And this partner opportunity had an RFP number. If you call the buyer, he will tell you the buyer called us. Part of the deal registration process is to submit detailed information, which we did: the buyer's name, department, phone number, etc. All that information is part of submitting for deal registration on-line.

"When NetApp says that, why don't they pick up the phone and call the buyer that called me? Nobody did. They can pick up the phone as easily as I did."


NetApp sales model is based on selling boxes, not long term relationships.

Thursday, July 12, 2007

When Jeff Browning was at NetApp we always paid close attention to what he said. Now he is at EMC, which is interesting. When Jeff speaks, it is still a good idea to pay attention:

Jeff has recently started a blog which is worth reading and it sheds some light on why things seem so confusing for NetApp's customers lately.

It seems to start with management -
Bruce Clarke (who has now left NetApp and probably does not mind my quoting him) once told me "NetApp is the best lead and worst managed company I have ever worked with." A sentiment with which I must admit that I agree. NetApp does the vision thing very well, no question. It's NetApp's implementation that kind of screws the pooch.
An example would be core ONTAP development. While I was there during the last part of my tenure, NetApp shipped ONTAP 7, which contained the functionality of flexvols. I was involved in the alpha testing of this product as it related to Oracle. I went to Engineering and asked about the path for migration from traditional volumes to flexvols, and was assured that there would be a tool to accomplish this built into the product. However, when ONTAP 7 shipped this tool had fallen off the truck, due to scheduling issues. This was never communicated to me (or pretty much anyone else from what I could tell) until after the product was about to ship.
The result was a train wreck. Many, many customers were justifiably upset when a feature which was a critical part of the ONTAP 7 release was denied them as they were marooned onto traditional volumes due to lack of ability to migrate. This problem became embarrassingly common.
Another similar issue was the creation of a version of ONTAP based upon the SpinOS, acquired with Spinnaker. I sat in an meeting where Engineering explained that they had "run out of seats at the game of musical chairs" which they called ONTAP development. In the threads of ONTAP 6.5, ONTAP 7, and the SpinOS based product, they basically had run out of capacity to develop. The result was the necessity to triage the development of a product line. The company made the bet on the SpinOS product, and dropped development of ONTAP 6.5, long before ONTAP 7 was ready for prime time. This resulted in yet more pain.
And the decision was made to ship the SpinOS product as a NAS-only version, and maroon SAN customers onto ONTAP 7 for an extended period of time after the ONTAP 7 product was ready to be replaced.


And he continues...

Another similar issue was the creation of a version of ONTAP based upon the SpinOS, acquired with Spinnaker. I sat in an meeting where Engineering explained that they had "run out of seats at the game of musical chairs" which they called ONTAP development. In the threads of ONTAP 6.5, ONTAP 7, and the SpinOS based product, they basically had run out of capacity to develop. The result was the necessity to triage the development of a product line. The company made the bet on the SpinOS product, and dropped development of ONTAP 6.5, long before ONTAP 7 was ready for prime time. This resulted in yet more pain.

This is really interesting.

Based upon these sorts of decisions, I took the position (which I still maintain), that NetApp would have been better off taking the millions they spent on Spinnaker out onto the parking lot in Sunnyvale, chunking them into barrels, soaking them with gasoline and torching them. Spinnaker was a net loss for NetApp in development energy and momentum, regardless of money. Spinnaker made false promises and created false expectations. All they added was confusion.

As I have said before many times " you can't make this stuff up!"

Wednesday, July 11, 2007

Fun with transportation

You might think from watching the FED EX and UPS commercials on TV, or from reading their brochures that transportation and shipping is easy and once everything is in the system it goes perfectly. With GPS in the trucks, and online databases what could possible go wrong?

Occasionally things just seem to be jinxed. No matter how many checklists we have, once things get in the hands of the freight business there is no certainty at all about where a shipment will end up or when it will arrive.

A few weeks ago FED EX delivered an overnight package destined for us to a woman two blocks away with Alzheimer’s disease. Why they delivered a package to a residential street two blocks away I can’t fathom. FED EX emailed us the signature and printed name and I looked in the phone book and got the lady’s home care provider to hold the package until one of our staff drove over to get the package. It was important legal documents. When a person from FedEx finally called me to say they couldn’t figure out where the package was they were stunned that I had called the name in the phone book, and we already had the package. High Tech solutions can’t replace a simple phone call and common sense.

Dealing with large shipments and airfreight can be frightening. A few years ago Eagle airfreight lost 6 cabinets with NetApp filers. How they could lose 6 refrigerator-sized cabinets still astounds me. But they lost them, and after months of wrangling gave us a check for $50.00 as insurance. Our insurance company had to cover the loss. We don’t use Eagle freight anymore.

International freight and transportation can get even more convoluted. Whenever you stick customs into the mix everything gets twice as hard. Using FedEx and UPS helps a lot. However, when shipping to England we have found that it makes no difference if you ship overnight on Thursday or Friday. Equipment arrives on Monday anyway.

You can spend hours on the phone with people who are all very nice, but getting a definite answer sometimes is impossible. Yesterday, I was helping a truck driver load up a shipment, and we were talking for a few minutes about the crazy job he has and the bizarre environment of transportation and freight. Before he left he said, “ If I did not laugh I would go crazy” He was right, but sometimes I need a beer to loosen up my sense of humor after dealing with these folks in the strange world of freight.

Friday, July 06, 2007

NetApp Costs Spiraling out of control? Zerowait has the answers, at a cost you can afford!

Earlier this week one of our oldest customers was telling me how tired he was of his NetApp sales & engineering team constantly trying to sell his company newer and more expensive storage and software. No matter how many times he tells them that he is happy with his current Filers, they turn the conversation toward the impending End of Availability of his equipment. Next they quote him draconian prices for his legacy maintenance and support. What he likes about Zerowait is that we are an independent company that helps him evaluate his actual costs of maintaining and managing his current storage infrastructure. With Zerowait service and support his storage planning is back in his hands instead of the pushy NetApp sales team’s. We allow him to concentrate on storage growth and management rather than on his vendor’s product release cycles and End of Life (EOL) announcements.

Successful NetApp system monitoring, maintenance & management can mean the difference between an efficiently run operation and one that’s grid-locked by vendor dictated equipment upgrades. With Zerowait on your side preventive maintenance and upgrade schedules can be based on usage thresholds defined by real world experience, and not by the manufacturer’s product release cycle, or the vendor’s sales quota.

Zerowait provides a simple, integrated set of services that can be tailored to your needs by department, by project, or by machine. Our flexible alert hierarchy allows you to automate preventive maintenance schedules based on thresholds you define. Zerowait offers a way to control your costs for acquisitions, upgrades and maintenance. Our customers told us they wanted a solution to out-of-control Network Appliance costs -- and we delivered.

Monday, July 02, 2007

Creating a global strategy one customer at a time.

Over the last few years it has become clear that our business grows mostly by reference and referral. NetApp customers, like all technology consumers, need to get all the value out of their equipment that they can over the long run. NetApp's sales force and technical sales engineers are evangelizing their new products to customers to sell product and make their quarterly numbers. Zerowait provides an independent, long term view which helps customers evaluate their current and future storage purchases strategically. Since Zerowait is an independent service and support business we can help customers evaluate their storage growth in light of their equipments specifications and work with hem to tweak their systems to get the most out of them.

Sometimes we speak to NetApp customers and they don't understand what we do. Imagine that! ;-) Zerowait provides independent monitoring, maintenance and system management to customers of NetApp equipment. Through our monitoring services we can provide insights on how to maximize your storage systems usage, and we also provide advanced parts replacement at a price much lower than the OEM. Our maintenance services include system upgrades, drive swaps and storage migrations and tuning. Our management services include everything from helping customers specify their new equipment to managing our customers storage infrastructures.

Zerowait provides the complete range of services to NetApp customers who are looking for an independent view on managing their storage investment. NetApp makes great hardware and software, Zerowait helps you get the most out of your IT investment over the long term.