On the eve of Structure 08 in San Francisco, the folks behind many of today’s cloud computing initiatives will be gathering at CloudCamp. GigaOM is one of the evening’s sponsors:

CloudCamp was formed in order to provide a common ground for the introduction and advancement of cloud computing. Through a series of local CloudCamp events, attendees can exchange ideas, knowledge and information in a creative and supporting environment, advancing the current state of cloud computing and related technologies.

If you’re in the city, come check it out. The event, which will be held from 5-9 pm on June 24th, is community-backed and free to attend. And it will be a great primer on cloud computing.

Want to know more about the rapidly changing Cloud Computing landscape? Preview our Cloud Computing Briefing or purchase the full version.

Adobe has decided to jump on the free consumer service bandwagon with the release of its new Photoshop Express online photo organizer and photo editor. Because Adobe also will store the photos (up to 2 gigabytes), it’s also the first big test of Adobe’s custom-built hosting infrastructure.

In a conference call to demo the Express software (which is really sweet vs. what’s on offer from the photo-editing software that comes with a digital camera, Picasa, or even displaying photos via Flickr) Doug Mack, VP of consumer and hosted solutions, said Adobe had built out a hosting infrastructure to support Express starting a year ago. He declined to go into the costs of the system, but said Adobe would be offering even more hosted applications in the future.

So I called Adobe for more information. Spokesman Geoff Baum declined to offer me much additional detail. He did, however, say the web-hosting infrastructure was entirely separate from Adobe’s internal IT, and that it was built to be reliable and to scale for many users.

Photoshop Express will be the first big test of the hosting platform, but in the next six to 12 months “more premium services within the same Photoshop.com web environment” will also be hosted. I’m guessing that in addition to paying for more storage, the flagship Photoshop program and Photoshop Elements might also find themselves with online services.

Since building out a solid hosting platform isn’t cheap, Adobe has plans to make it pay beyond just charging a bit more for online storage. Baum said Adobe will make the investment pay through upselling services, potential advertising, and OEM deals with other companies, such as one in which a company licenses a component of the Photoshop Express software.

The services hosted on the newly built infrastructure may not be making money today, but it’s clear Adobe has gotten the web services religion and plans to integrate those services into its popular software packages. “This is an important direction for the company and something we wanted to control and scale as necessary,” Baum said. “[Not outsourcing] it is mainly an indication that hosted services are important to what we’re doing.”

It’s an ethos that I hope will play out across Adobe’s other software programs, especially given the potential for competition from companies such as iScribd. When Adobe launched its hybrid rich Internet application development platform AIR, CTO Kevin Lynch told Wired:

“The other effect is hosted services. Software is moving from being packaged, where you develop for a particular operating system and put it in a box, to being developed and distributed over the internet and being designed to run across operating systems. That’s where all the innovation has moved to. Software isn’t as OS-specific anymore, it’s moving to rich Internet applications. It’s a sea change in how software in general is being built.”

Adobe is clearly preparing for that sea change within its own product portfolio.

On-demand computing promises two things. One, the ability to grow or shrink capacity based on need. And two, the ability to drag and drop virtual machines instead of racking and stacking physical ones.

With today’s on-demand services, although the machines are virtual, they still need to be defined and managed by real people, one server at a time. As a result, much of the expected savings from virtualization never really materializes. Elastra, a San Francisco-based startup backed by Hummer Winblad Venture Partners, wants to change this by letting IT teams provision entire application clusters of whatever software they choose, into any on-demand computing platform, automatically.

Here’s how it works: Elastra uses a pair of document formats that describe what the application does and how it does it. Feed both into the company’s “cloud server” and you get a virtual data center, complete with monitoring, accounting and automated deployment, running on the virtual infrastructure of your choice.

Fixing IT with markup languages has been done before. The Data Center Markup Language was one cross-vendor attempt to describe data center environments and the dependencies between components. But Elastra CEO Kirill Sheynkman says his company’s different: Because virtual machines are software, IT teams can not only use Elastra’s language to describe their application cluster, they can also “run” that description and provision the application in its entirety on whatever virtual infrastructure they want.

The company’s first release, launched today, provides a relational database atop Amazon’s EC2/S3 platforms. “We did database first because it’s challenging enough to be interesting, but you’re not solving the world,” Sheynkman said. Eventually, the company plans to support LDAP servers, databases, message buses, search engines and other application building blocks. Elastra could also pave the way for licensing and compliance of enterprise software running on demand. “Commercial software ISVs (independent software vendors) have a problem running in the cloud,” he said. “It’s fairly simple to put a license server in there.”

Elastra doesn’t care which virtual infrastructure you use: It wants to separate the underlying virtual computing infrastructure from the task of system design, deployment and administration. Some firms will want to run their own internal cloud, too, and the company has built its cloud server with that in mind. “Lots of big companies want to offer a public compute utility,” said Sheynkman, “but there are also private companies big enough to do it on their own terms.”

The industry is gradually moving from virtual machines to virtual application clusters. By turning a description of the cluster into running code on the computing infrastructure of your choice, companies like Elastra get IT one step closer to the true potential of on-demand computing.

Right now Microsoft’s Windows Live efforts are the software giant’s answer to web applications and cloud computing. In fact, however, they’re less a cloud strategy than a layer of fog over the multibillion-dollar packaged software franchises that keeps Microsoft going.

But the Redmond-based behemoth isn’t dumb, and as its chief software architect, Ray Ozzie, said in his interview last week with Om, Microsoft is beginning to navigate the cloud. The release of a new version of FolderShare points to that. The Austin-based startup that built the product was acquired by Microsoft two and a half years ago, only to undergo more than four reorganizations within the first 18 months. In fact, the FolderShare product was going to be shelved — that is, until Microsoft realized that web applications were the place to be. So it’s taken until now to launch a new version.

If you read our review, you know my complaint about FolderShare is that it’s not really doing much in the cloud, which makes it difficult to use for online collaboration and backup. However, judging from the design elements and team members working on FolderShare and Microsoft’s online storage product, SkyDrive, I’m betting we’ll soon see another new launch, one that brings the two together. While I still plan on checking out some of the online storage programs mentioned in the comments (isn’t ElephantDrive simply a repackaged Amazon s3?), I’m also eager to see what Microsoft can do.

These days, San Francisco’s skyline is not marked so much by the Transamerica Pyramid or Coit Tower as it is by cranes and the jagged outlines of half-finished buildings, symptoms of its struggle to meet the growing demand for housing and office space. The city by the Bay’s construction activity, however, only mirrors the frenzied buildout of the web’s infrastructure.

From Google to Facebook to Hulu to MySpace — web businesses are seeing unprecedented growth. In an effort to meet the demands of their customers, these companies have been forced to spend billions of dollars on servers, switches and a whole lot of other stuff that, taken together, forms an “infrastructure.” But such unprecedented growth is putting that infrastructure under extreme stress.

A parallel to this buildout took place in the mid-1990s, when the emergence of the Internet economy resulted in booming demand for bandwidth, connectivity and other such services. That led to an unprecedented spending cycle, with telecoms sinking billions of dollars into new technologies that helped spawn startups such as Ciena and Juniper Networks.

The new web infrastructure buildout presents a similar opportunity, both for entrepreneurs and their backers. As I noted not too long ago, we are in a period of flux: The platforms on which we have done business for over a decade are starting to provide diminishing returns; the smart money, meanwhile, is seeking new platform structures.

We’ve already started to see some cool new ideas. We plan to explore these ideas, and their manifestations — which range from new databases to new architectures for data centers — at our first GigaOM conference, Structure 08.

Structure 08 will gather the most innovative and influential industry leaders together to explore the latest Internet infrastructure buildout. It will sort through the emerging and disruptive computing technologies and inform businesses on how best to leverage them. And it will provide insight to investors and executives on the best implementations, ideas and startups out there today — and what to look for tomorrow.

The conference will take place on June 25th at the Mission Bay Conference Center in San Francisco. Among our scheduled keynote speakers are Werner Vogels, CTO of Amazon.com, and Greg Papadopoulos, CTO of Sun Microsystems. Other speakers include VMware co-founder Mendel Rosenblum and Salesforce.com founder Parker Harris.

I am working with Surj Patel and our stellar program committee on this conference. We are still in the process of finalizing the line-up, panels and speakers so in the meantime, your suggestions are more than welcome. Please drop me a line with speaker nominations and panel suggestions. If you would like to buy early-bird tickets, please visit the conference web site.

Hope to see you on June 25th.

That was the question asked of the four panelists on Monday’s Scalability Boot Camp Panel at South by Southwest. The panelists, who represented various consumer sites, all said that at some point in their online ventures the answer to that question was no. As a result they’ve ended up learning how to build network architectures that can support a large number of users.

One way (are you listening Yahoo?) was to restrict launches by making them available to a limited amount of people over a fixed period of time, because no one server can handle all the users of a site hopping on all at once. Another was to figure out what your users want before trying to figure out your network architecture. Blaine Cook of Twitter confessed that when they launched their service, no one was sure what people would use it for, calling it the worst idea ever.

All the talk about failure led to a discussion about how scalability problems hurt or enhance a company’s reputation. Cook said the press mentions of Twitter’s downtime as a plus, but the irritated users were obviously a minus. Sandy Jen, co-founder of IM service Meebo, echoed comments made by Jakob Heuser of Gaia Online when she talked about letting users know what’s going on while the site is down.

She also emphasized the importance of getting the business side involved with the technical and operations side. “No feature gets out the door without talking to operations” at Meebo, she said.

The audience members seemed most concerned in how to get the business side talking about scalability issues, while the panelists, all of whom had technical backgrounds, talked about focusing on the user experience and trying to translate down time into money. All good advice, but I imagine that conversation is much easier to have at a startup led by a tech-savvy CEO.

After the debacle of yesterday’s on-stage interview with Mark Zuckerberg at South By Southwest, I decided to take myself out of the process as much as possible by offering up a Q&A. The results below are heavily edited excerpts from my conversation with the Facebook CEO. Instead of grilling him too much about monetization and the company’s $15 billion valuation, we spent most of the time talking about infrastructure, scaling Facebook’s architecture and what the ad platform is going to look like. And I’ll tell you right now: When I asked about money, my questions were quickly shot down.

ME: What does the Facebook infrastructure look like right now as it supports more than 67 million users?

ZUCKERBERG: I can’t talk about the number of servers specifically, but I can say it’s not on the order of hundreds of thousands yet; it’s in the order of tens of thousands. Would you like to talk about how the architecture has evolved over time?

It’s pretty fascinating to see that the evolution of the network architecture pretty much reflects the evolution of the site. When we were in a few colleges one of the theories there was that most of the people would want to talk to people at their own schools, so we set up a network architecture based on the school. We had a Harvard database and web server and routed traffic to them through sub-domains. As time went on, and we reached a reasonable scale, we had to break out into tiered services.

In mid-2005 or so we also started relying heavily on memcached, and today it’s become such an important part of our infrastructure that we’ve actually become the head developers on one of the branches of the code.

At this point in Facebook we’re now running our networks and have multiple data centers across the country, one on the West Coast and one on the East Coast. So we’re working on problems of reducing latency in transferring data from one to the other.

ME: So as you expand internationally will Facebook need an international data center? When?

ZUCKERBERG: That will be something at some point.

ME: Do the applications running on Facebook affect the network? How much of that does Facebook need to support?

ZUCKERBERG: We have an API tier that serves the data requests, but the same servers serve the requests generated inside of Facebook and those form outside of Facebook. We’re trying to blur the boundaries of content created inside Facebook and applications created outside Facebook, and that’s reflected in the architecture.

ME: What challenges will you guys need to address as you scale?

ZUCKERBERG: We put a ton of focus into making sure going forward that our development speed will be very fast. But you know the phrase, “Premature optimization is the root of all evil”? If we spend our time trying to optimize our architecture it could end up being one that’s difficult to develop on and going forward, a big focus for us is making sure that’s not the case.

One of the recent places is internationalization. The infrastructure to do that is pretty fascinating but it has the potential to slow down future development as people ask if they need to develop an application in another language first, so we’ve basically built a wrapper for every string of text and the developer puts a wrapper around that text and we send it to users to translate. So that doesn’t slow down the development speed.

ME: How do you handle quality control?

ZUCKERBERG: In the three cases we’ve done so far (Spanish, German and French), the quality of the translations that we got was actually better. If you wanted to get professional translators it wouldn’t be scalable.

ME: Let’s talk about monetization. You said yesterday that you envision the social advertising landscape evolving over the next 10…15…20 years. How will those ads evolve and when will we start seeing aspects of them on Facebook? And where does Beacon, which you said wasn’t an ad effort, fall into this?

ZUCKERBERG: Beacon was a part of the platform. It was part of this while effort to blur the boundaries between what’s inside Facebook and what’s outside Facebook. Beacon was our first cut at a protocol to do that.

When it comes to social ads we really want to line up what people are trying to do on Facebook and the utility it offers with monetization. If you look at what people are trying to do on the site, it’s communicating and connecting with each other and sharing information, so the business model should be around people sharing information and staying connected.

In banner advertising, people who have developed a trust with the audience run a banner ad and the trust bleeds over to the ad so people pay attention to it.

ME:(making a skeptical face.)

ZUCKERBERG: What? You look like something’s wrong? People go to a content site to see a specific kind of content and will trust those ads relate somehow to it. On Facebook, people aren’t coming to see content from Facebook; they’re coming to see what other people are sharing, so the most natural analog would be having the ads be information shared among the people. Because so much of our society has some commercial component it seems like there will be a way to both share information and line that up with what advertisers want.

Some amount is happening as advertisers pay to accelerate that distribution of information. The amount they’d be willing to pay is proportional to how much it is accelerated.

ME: And before we leave I wanted to ask about revenue and the fact that you mentioned yesterday that you are operating at breakeven.

ZUCKERBERG: I said we’re operating around breakeven. In the past we’ve run at sometimes more and sometimes less, but as a private company we don’t disclose those numbers. A lot of people ask us what we’re going to do with that Microsoft money, but we have no specific purpose for it. We didn’t take that money to buy something.

Fresh from his Mix’08 keynote, Microsoft’s Chief Software Architect and industry luminary, Ray Ozzie, spent some time on the phone with me, discussing everything from the company’s services strategy, to the economics of cloud computing, to the relevance of desktop and infrastructure challenges. What follows is a highly edited version of our 20-minute conversation.

Enjoy this interview, the first of what I hope will become a series of conversations with tech greats.

OM MALIK: You outlined Microsoft’s software-plus-services strategy, but what I want to know about is the changing role of the desktop in this service’s future.

RAY OZZIE: I think the real question is (that) if you were going to design an OS today, what would it look like? The OS that we’re using today is kind of in the model of a ’70s or ’80s vintage workstation. It was designed for a LAN, it’s got this great display, and a mouse, and all this stuff, but it’s not inherently designed for the Internet. The Internet is this resource in the back end that you can design things to take advantage of. You can use it to synchronize stuff, and communicate stuff amongst these devices at the edge.

A student today or a web startup, they don’t actually start at the desktop. They start at the web, they start building web solutions, and immediately deploy that to a browser. So from that perspective, what programming models can I give these folks that they can extend that functionality out to the edge? In the cases where they want mobility, where they want a rich dynamic experience as a piece of their solution, how can I make it incremental for them to extend those things, as opposed to learning the desktop world from scratch?

OM: So basically you’re saying that in this new environment, that you have to give up on your legacy of desktop and just view the world from a web perspective?

RAY OZZIE: Well, I can’t say give up on it. Here’s the way I talk about it to people at Microsoft. The desktop is very useful. People use it a lot on a daily basis. There are things that the web is good for, but that doesn’t necessarily mean that for all those things that the desktop is not good anymore. What I think is important is to re-pivot the center of what we are trying to accomplish.

OM: What makes you guys think that you can actually do better than everybody else?

RAY OZZIE: I’m not going to get cocky. The reason I’m a believer is that Microsoft as a company is in a number of different markets. I think we’re well positioned, because we have a selfish need to do these things, and because we have platform genetics. We have the capacity to invest at the levels of infrastructure that are necessary to play in this game. So I think we’ll be well positioned. I can’t tell you specifically which aspects we’re going to kick somebody else’s butt, or where they’re going to kick our butt, but I think we’re pretty well positioned.

OM: I buy into the whole services model, but then I see what happened a couple of weeks ago — Hotmail goes down for quite a long time. And this happens way too often, not only just at Microsoft, but at other services also. It is very hard for me to imagine, we keep talking about services, but the reliability of the infrastructure is just not there.

RAY OZZIE: It’s not straight engineering, and it’s not an art. It’s somewhere in-between. And we are all learning. And so if you look at the innards of a Yahoo or a Microsoft, an MSN, or a Google, you will see the people who have designed the systems and have taken a number of the things we’ve learned in the enterprise space. We have to throw them them away, because the way that we did it in the enterprise space was more tightly coupled. We need to be more loosely coupled.

So I’m not going to make any excuses for downtime. We need to develop more and better application design patterns that we give to developers that let them develop mesh-oriented apps at birth, horizontal apps that can suffer massive failures of certain aspects of their infrastructure, while still surviving.

OM: It (mesh-oriented apps) sounds like a great idea, but in reality can we actually deliver that kind of a mesh app architecture, and how soon?

RAY OZZIE: I think that you’ll see is over the course of this year, to 18 months, you’ll see the incumbents and startups, both, do their first big volleys of services platform, apps tools, runtimes, various things. It really isn’t being taken seriously right now by anybody except Amazon. They’ve done the world a service by putting out there some fairly provocative, interesting services.

OM: The costs of computing, hardware and bandwidth are dropping quickly. Do you believe that the cost will come down fast enough to make cloud computing actually a profitable business?

RAY OZZIE: Well, it’s unlikely that we would get into it if we didn’t think it was going to be a profitable business. So we’ll just manage it to be profitable. It’s going to have different margins than classic software, or the ad (-supported) business. But, we have every reason to believe that it will be a profitable business. It’s an inevitable business. The higher levels in the app stack require that this infrastructure exists, and the margins are probably going to be higher in the stack than they are down at the bottom.

OM: Can you actually elaborate a little bit on that, like when you say higher in the stack, what precisely do you mean?

RAY OZZIE: Let’s go all the way up. Let’s stick to boring old enterprise, all the way up at business solutions, HR apps, or things like that. Somebody who is selling those apps is going to build in, more than likely, the underlying utility costs within their higher-level service. It will still be cheaper to do those things on a service infrastructure than it is on a server infrastructure, but the margins will still be higher to people who build solutions that customers understand the business value of.

When you go down to selling bandwidth, or selling MIPS there will be competition at that level. So the margins, at generic commodity levels are going to be substantially lower.

OM: When do you think utility computing can be a profitable business; are we’re looking at like maybe two years, four years out before it actually starts to become a profitable entity?

RAY OZZIE: (Let’s) take (one company) who is in the market today: Amazon. They chose a price point. There are either customers at that price point or not. They may have priced themselves at expected costs as opposed to actual today costs, but it doesn’t really matter. They could have brought it out at twice the existing price and there still would have been a customer base, and they’d be making money at birth.

I think all of these utility-computing services, as they’re born will either be breaking even or profitable. At the scale that we’re talking about, nobody can afford, (even Microsoft) can’t afford to do it at a loss. We could subsidize it, I suppose. Google could subsidize it by profits in other parts of their business, we could subsidize it, but I don’t think there’s any reason that any of us in this world would bring out that infrastructure like this without charging for what we’re paying, and then trying to make some profit over it. The cost base is so high in terms of building these data centers you do want to kind of make it up.

Written by Geva Perry, chief marketing officer at GigaSpace Technologies.

We are witnessing a seismic shift in information technology — the kind that comes around every decade or so. It is so massive that it affects not only business models, but the underlying architecture of how we develop, deploy, run and deliver applications. This shift has given a new relevance to ideas such as cloud computing and utility computing. Not surprisingly, these two different ideas are often lumped together.

What is Utility Computing?

While utility computing often requires a cloud-like infrastructure, its focus is on the business model on which providing the computing services are based. Simply put, a utility computing service is one in which customers receive computing resources from a service provider (hardware and/or software) and “pay by the drink,” much as you do for your electric service at home – an analogy that Nicholas Carr discusses extensively in “The Big Switch.”

Amazon Web Services (AWS), despite a recent outage, is the current poster child for this model as it provides a variety of services, among them the Elastic Compute Cloud (EC2), in which customers pay for compute resources by the hour, and Simple Storage Service (S3), for which customers pay based on storage capacity. Other utility services include Sun’s Network.com, EMC’s recently launched storage cloud service, and those offered by startups such as Joyent and Mosso.

The main benefit of utility computing is better economics. Corporate data centers are notoriously underutilized, with resources such as servers often idle 85 percent of the time. This is due to overprovisioning — buying more hardware than is needed on average in order to handle peaks (such as the opening of the Wall Street trading day or the holiday shopping season), to handle expected future loads and to prepare for unanticipated surges in demand. Utility computing allows companies to only pay for the computing resources they need, when they need them.

What is Cloud Computing?

Cloud computing is a broader concept than utility computing and relates to the underlying architecture in which the services are designed. It may be applied equally to utility services and internal corporate data centers, as George Gilder reported in a story for Wired Magazine titled The Information Factories. Wall Street firms have been implementing internal clouds for years. They call it “grid computing,” but the concepts are the same.

Although it is difficult to come up with a precise and comprehensive definition of cloud computing, at the heart of it is the idea that applications run somewhere on the “cloud” (whether an internal corporate network or the public Internet) – we don’t know or care where. But as end users, that’s not big news: We’ve been using web applications for years without any concern as to where the applications actually run.

The big news is for application developers and IT operations. Done right, cloud computing allows them to develop, deploy and run applications that can easily grow capacity (scalability), work fast (performance), and never — or at least rarely — fail (reliability), all without any concern as to the nature and location of the underlying infrastructure.

Taken to the next step, this implies that cloud computing infrastructures, and specifically their middleware and application platforms, should ideally have these characteristics:

• Self-healing: In case of failure, there will be a hot backup instance of the application ready to take over without disruption (known as failover). It also means that when I set a policy that says everything should always have a backup, when such a fail occurs and my backup becomes the primary, the system launches a new backup, maintaining my reliability policies.
• SLA-driven: The system is dynamically managed by service-level agreements that define policies such as how quickly responses to requests need to be delivered. If the system is experiencing peaks in load, it will create additional instances of the application on more servers in order to comply with the committed service levels — even at the expense of a low-priority application.
• Multi-tenancy: The system is built in a way that allows several customers to share infrastructure, without the customers being aware of it and without compromising the privacy and security of each customer’s data.
• Service-oriented: The system allows composing applications out of discrete services that are loosely coupled (independent of each other). Changes to or failure of one service will not disrupt other services. It also means I can re-use services.
• Virtualized: Applications are decoupled from the underlying hardware. Multiple applications can run on one computer (virtualization a la VMWare) or multiple computers can be used to run one application (grid computing).
• Linearly Scalable: Perhaps the biggest challenge. The system will be predictable and efficient in growing the application. If one server can process 1,000 transactions per second, two servers should be able to process 2,000 transactions per second, and so forth.
• Data, Data, Data: The key to many of these aspects is management of the data: its distribution, partitioning, security and synchronization. New technologies, such as Amazon’s SimpleDB, are part of the answer, not large-scale relational databases. And don’t let the name fool you. As my colleague Nati Shalom rightfully proclaims, SimpleDB is not really a database. Another approach that is gaining momentum is in-memory data grids.

One thing is certain: The way the industry has traditionally built software applications just won’t cut it on the cloud. That’s why companies such as Google, Amazon and eBay have developed their own infrastructure software, opting not to rely on products from the large middleware vendors such as Oracle and BEA, who designed them with a very different approach in mind.

For this reason, we are seeing the emergence of a new generation of application platform vendors. These vendors, which include my own company, GigaSpaces, are building software platforms made for the cloud from the ground up: “cloudware,” if you will.

So although they are often lumped together, the differences between utility computing and cloud computing are crucial. Utility computing relates to the business model in which application infrastructure resources — hardware and/or software — are delivered. While cloud computing relates to the way we design, build, deploy and run applications that operate in an a virtualized environment, sharing resources and boasting the ability to dynamically grow, shrink and self-heal.

Google is buying a piece of a new transpacific fiber optic cable, according to research firm TeleGeography. This will be yet another piece of what can be loosely described as GoogleNet, a fiber network built and leased by the search engine and advertising giant to meet its ever-growing bandwidth requirements. Google is one of the six investors in the “Unity” undersea cable that will connect the U.S. and Japan. The new cable is going to be built by Tyco Telecommunications and NEC for about $300 million.

TeleGeography says that Google has been trying to buy a piece of a transpacific cable for a while now. Google CEO Eric Schmidt had admitted to as much a few months back. I have written about how Google is using its infrastructure (including network) as a strategic advantage, and this latest move is an extension of that philosophy. It has been buying dark fiber to grow its network, as I had first reported back in 2005.

TeleGeography estimates that transpacific bandwidth is eight times higher than transatlantic routes. Google can now get capacity at cost, and it can also squeeze more out of its infrastructure. Our good friend Alan Maudlin, TeleGeography research director, doesn’t think this is going to start a trend.

“While Google is the first non-telecom company to take an active role in ownership of a submarine cable, it’s not likely that this is the beginning of a new trend,” commented TeleGeography Research Director Alan Mauldin. “Although many non-telecom companies have high bandwidth requirements, few will venture into owning submarine cables anytime soon.”

Update: Google’s manager of network acquisitions, Francois Sterin, on their blog writes about why they chose to invest in the Unity cable.

As more and more people conduct online searches and interact with applications like Gmail, Google Earth and YouTube, we’ve had to think outside the box to create a more scalable, affordable and easy to manage network that meets our users’ needs worldwide. One of the biggest challenges we face is staying ahead of our broadband capacity needs, especially across Asia.

This is the first time Google has admitted to building and buying fiber to build their own network. I have often received denials from their PR folks, but I guess my sources were better.

If you’re wondering whether we’re going into the undersea cable business, the answer is no. We’re not competing with telecom providers, but the volume of data we need to move around the world has grown to the point where in some cases we’ve exceeded the ability traditional players can offer. Our partnership with these companies is just another step in ensuring that we’re delivering the best possible experience to people around the world.

Update #2: Google press release has some details about the Unity Cable.

The Unity consortium is a joint effort by Bharti Airtel, Global Transit, Google, KDDI Corporation, Pacnet and SingTel. The name Unity was chosen to signify a new type of consortium, born out of potentially competing systems, to emerge as a system within a system, offering ownership and management of individual fiber pairs.

This new 10,000 kilometer (km) Trans–Pacific cable will provide connectivity between Chikura, located off the coast near Tokyo, to Los Angeles and other West Coast network points of presence. At Chikura, Unity will be seamlessly connected to other cable systems, further enhancing connectivity into Asia.

As someone whose job involves understanding how certain people and things relate to one another, the idea of the semantic web is both compelling and scary. It could make my job that much easier, or it could make me as redundant as switchboard operators are today.

Coding information in a standard way so that machines can see how one person relates to another, or how a string of words could alternately be a movie or a book title, is a challenge. But plenty of companies are taking little bits and pieces of the problem and solving them. One such startup, Radar Networks, the maker of Twine, today received $13 million in funding from Velocity Capital, Vulcan Capital and DFJ. Other startups such as EVRI and Freebase have also benefited from VC interest in the semantic web.

Some of the companies are following the standards offered by the W3C, which is pushing RDF as a standard data structure to underlie the semantic web. But not all companies working on helping machines figure out the relationships and categories that most humans have learned use that standard.

Nor are all the companies interested in making the semantic web work startups. Yahoo uses RDF in some of its offerings and Google’s efforts with its social graph API initiative resembles the semantic web in its goals. Instead of using RDF, however, it’s using XFM and FOAF tags.

Reuters is another company that sees potential is getting machines to understand relationships. Earlier this month its CEO laid out a pretty compelling vision (at least to Tim O’Reilly) about how Reuters would rely less on delivering information and more on packaging its information in a way that could be used by analysts and computers to quickly delineate relationships.

Reuters would then be able to take its content, make it programmable and offer that data to users, who could then do with it what they will. Things like making relationship charts that currently can take a journalist and graphics department a couple of days to complete, and must then be monitored and changed manually, become easy.

The effort to render all of the data on the web into a semantic form will take a while. Nova Spivack, CEO of Radar Networks, believes that semantic web applications are currently in the early adopter phase. Twine will unveil its efforts in March through a private beta and another startup, AdaptiveBlue launched a semantic plug-in called Blue Organizer earlier this month. Spivack believes that in 2010 mass adoption will take place as people start to expect machines to make “intelligent” connections between people and things.

All of this is interesting, but putting a layer of semantic code over the existing web raises some concerns. One is the danger of inaccurate or at the very least less nuanced sense of relationships between people. Another is the everlasting nature of information on the web. How will coded tags be able to follow the intricacies of human relationships as fights ensue, jobs shift and even names change?

Another issue that we’ll have to deal with is confusion as people try to figure out what the semantic web really is. I’m thinking of it as code added to existing and new web content that helps determine and maybe track relationships between people and contexts for objects. I’m not married to the W3C standards, however, and others are doing this without using those particular programming tools.

There are also plenty of other definitions and hopes for the next phase of the web that may play out before we get an intelligent Internet. It’s already apparent that the web will continue to become more useful over time, but won’t ever replace the benefits of human interactions. If you doubt me, just recall your most fulfilling customer service call with a person compared with your most fulfilling experience with an automated agent. While both are helpful, sometimes you need a real, live human being.

Google, according to The Wall Street Journal is thinking about teaming up with Space Data Corp., a company that sends balloons carrying small (micro) base stations about 20 miles up in the air for providing connectivity to truckers and oil companies. The balloons burst almost every 24 hours and need to be sent up again and again. The electronic payload is retrieved by farmers after it drifts back using a small parachute. The farmers do it because they get $100 per payload retrieved, WSJ says.

The Internet giant — which is now pushing into wireless services — has considered contracting with Space Data or even buying the firm, according to one person…Google believes balloons like these could radically change the economics of offering cellphone and Internet services in out-of-the-way areas, according to people familiar with its thinking.

Given that there is only one anonymous source that is linked to Google, I remain highly skeptical of this plan. And if it is true, then it is yet another example of Google having more money than knowing what to spend it on, which in itself is a dangerous sign for its investors.

Amazon’s foray into on-demand web services such as its S3 storage and EC2 computing service has done two things: brought the focus onto cloud computing and at the same time up-ended the pricing structure of the Internet infrastructure. Of course, it has also prompted a spate of copycats and liberal interpretation of the term “cloud computing.”

Take Mosso, an on-demand hosting start-up backed by Rackspace of San Antonio, Texas, as an example. For $100 a month, Mosso is offering 50 GB disk-space and 500 GB bandwidth and 3 million web requests per month. Beyond that it is 25 cents per gigabyte for bandwidth and 3 pennies per 1000 requests.

Mosso Co-founders Todd Morey and Jonathan Bryce, both ex-Rackspace employees, are going after startups and web app developers with their new “hosting cloud.”

While Mosso seems to be an easy enough service to use, with a simpler pricing structure, it is hard for me to think of it as a “cloud computing” service like the way I think of Amazon Web Services. AWS, for example, charges for what you use. There is no upfront monthly fee, a business practice common to hosting services. Moreover AWS offers pieces of the infrastructure — computing, storage, etc. — on demand, not as a packaged solution.

From that perspective, Mosso is competitive with Joyent, a Marin, Calif.-based startup that recently started offering such scale-as-you-need-to services to Facebook application developers. That said, I think Mosso-type services are good for the hosting business, and also for startups and web app developers, since they provide a simpler pricing model along with the ability to scale.

Amazon’s S3 cloud-based storage service went down earlier this morning, according to numerous tips we’ve received. The service has impacted many companies, including folks like Twitter. According to our tipsters, the service went down around 4:30 a.m., and is showing a 500 Internal Server Error message.

Amazon Web Services forums are full of people chatting about the outage. One poster on the forum summed up the situation nicely, saying, “The s3 service is great but this just proves you can’t rely on it, this is a major issue especially since it’s been down for so long. Way to go Amazon.”

This outage, one of the first large-scale problems to hit Amazon, shows that a lot of work needs to be done before we can completely rely on the cloud. As I have often said, we are running the 21st century web on infrastructure that was dreamed up in the 1990s, long before the web’s current scale. Still, that doesn’t take away my long-standing enthusiasm for Amazon’s web services strategy.

We will keep you posted. Meanwhile, let us know how you have been impacted and what you are doing to build the redundancy of your web service.

Nick Carr has his take on the situation. “Given that entire businesses run on S3 and related services, Amazon has a particularly heavy responsibility not only to fix the problem quickly but to explain it fully,” he writes. I agree with him, and hopefully Amazon will do the needful. Amazon says it is fixed it, but there seem to continuing problems with the service, as the forum indicatess.

Written by Greg Olsen, Founder and CTO of Coghead

Imagine that your friend tells you that he has an idea for a new Palo
Alto, Calif.-area restaurant that he wants you to invest in. His pitch goes like
this: The restaurant is all about self-sufficiency. In addition to
actually serving good food, this restaurant will feature the following:

• All food served will be organically raised and processed on-site
• Power will be provided by an on-premise power plant
• Water will be provided by a well-and-rain capture
• A self-contained waste management system will eliminate the need for a
  sewer hookup

While there are probably some people in Palo Alto that might actually
think this is a good idea, you being of sound mind respond, “Is this a
joke? Why build basic infrastructure like foodstuff production, water,
sewer, etc. when very efficient, cost-effective, ‘pay-as-you-need-it’
options already exist?”

Imagine that your other friend tells you that she has an idea for a new
software application company she wants you to invest in, and that
this company (in addition to actually creating a useful service-based
application) will:

• Build and manage redundant data centers with a carefully constructed
  custom hardware and software stack
• Set up an advanced network peering infrastructure for redundancy and
  improved latency
• Implement a flexible payment system for customers and channel partners

This could also be misconstrued as a joke. Why would a small
application provider spend so much capital, time and energy building
infrastructure when readily available ‘pay-as-you-need-it’ services
exist, such as compute, storage and network infrastructure services (e.g. Amazon’s
EC2 & S3 services), and payment services from Google, Amazon, etc.?

It is possible that specifics of your friend’s application make use of
available service options infeasible, but it is just as likely that your
friend has simply not yet adapted to a service-based infrastructure
reality. There are always seemingly good reasons to continue doing things the way they were done in the past, and transition
always presents challenges. As ironic as it may be, we continue to see
software applications deployed as a service but which fail to use any
service-based infrastructure themselves. They are two basic reasons for
this situation: Change of existing operational services is hard. So is changing people behavior.

Once an application service is deployed, infrastructure changes are
hard to make. Often commitments of capital cannot be undone without very high
switching costs, such as advanced purchases of compute and storage
capacity. Many architectural choices can have lasting ramifications.

For example, if a provider built their application based on the
assumption of very large SMP servers, a proprietary commercial database
clustering approach and vendor-specific HA infrastructure, they would
find it difficult if not impossible to move to a service-based
infrastructure that’s based on generic hardware/software platforms and
horizontal scaling.

Even for a new application service, it’s often hard to find people who
will embrace disruptive infrastructure options. It is almost inherent
in human nature that once we develop a difficult skill we are reluctant
to give up using it — even after simpler and more efficient alternatives
become obvious. Often, people perceive that their livelihood is tied to
the skill and then fear their own obsolescence in the obsolescence of
the skill. The history of software applications provides a rich set of
examples of this phenomenon. At one time, it was common for software
application providers to create their own hardware, operating systems,
networking infrastructure, languages, compilers, user interface
technology, etc. Eventually, successful application providers took
advantage of standard hardware platforms, operating systems and
languages — to the detriment of the many providers that clung to the prior
model. Likewise, vendors that leveraged the Internet and application
servers gained, while many others continued to cling to proprietary
client/server architectures and were the worse for it.

The recent “software-as-service” phenomenon is a particularly
interesting example of disruptive change. SaaS was first seen as a
disruptive force inside of the IT groups of large application users.
Most companies are starting to understand that they would be better off
with less information technology on their premises and more of it
procured as a service over the Internet. Still, however, many within IT
organizations are reluctant to embrace this form of change. (Personally, every time I see an IBM Blade Server commercial during a major sporting event, I’m wondering what percentage of the viewers know what it is,
what percentage of those could actually affect the purchase of one, and
then what percentage of those should actually be buying data center
servers in an efficient universe).

We are now at point where implementors of SaaS capabilities are being
disrupted by newer SaaS capabilities. Services that are built largely
from other services are a reality, and offer many clear advantages. The
types of services that could be used in the creation of new services
span the spectrum, from base infrastructure services to complementary
high-level application services that can be composed or mashed up.
Example services include: compute and storage services; DB and
message-based queuing services; identity management services; log
analysis and analytic services; monitoring and health management
services, payment processing services; e-commerce services like
storefronts or catalogs; mapping services; advertisement services; in
addition to the more well-known business application services like CRM
and accounting.

The move to SaaS applications built on SaaS is a much more profound
shift than the move from on-premise applications to SaaS applications.
The software industry is beginning to display characteristics that
mimic the supply chains and service layering that are commonplace in
other industries like transportation, financial services, insurance,
food processing, etc. A simple set of categories like applications,
middleware and infrastructure no longer represents the reality of
software products or vendors. Instead of a small number of very large,
vertically integrated vendors, we are seeing an explosion of smaller,
more focused software services and vendors. The reasons for this
transition are simple: It takes less capital and other resources to
create, integrate, assemble and distribute useful software
capabilities.

By leveraging service options like Amazon’s EC2 and S3, a
small company can deploy a complex, highly available and scalable
multi-user software application — without huge upfront investments in
hardware or software infrastructure. Likewise, a very small company can
build a simple, narrowly focused service and can cost-effectively sell
it to a mass audience. Neither of these companies would have been possible only a short time ago.

A new software service economy is rapidly unfolding and is causing
disruption in the software industry. Ironically, some of the first
victims of this new economy may be some pioneers of the software-as-a-service movement. Today, many established SaaS application providers
are applying much more of their precious focus and capital to
infrastructure issues than newer competitors that are aggressively
utilizing service-based infrastructure. The self-contained restaurant
and the build-it-all-ourselves SaaS application vendor both have
seemingly good rationales for their chosen paths, but both will
ultimately end up as anachronisms that are left behind by their competition.

Digital minimalists got a big lift this morning when Microsoft, Google, VeriSign and IBM all said they’d begin supporting OpenID, the open-source standard that seeks to enable consumers to use a single identity and password to log in to any web site on the Internet.

The idea of a universal ID to simplify consumer’s lives has been around for some time. But many companies have been unwilling to let go of that last connection to their end users, the one-off login required to access each site. With today’s endorsement, OpenID’s potential base has been pushed to nearly 1 billion users.

And OpenID is proving to be a huge windfall for at least one startup: Tulsa, Okla.-based Vidoop, which we introduced here in October. Vidoop’s consumer play, called MyVidoop, uses images rather than letters or numbers to create OpenID logins. The images are handy in one critical way: Vidoop sells them to advertisers.

Vidoop has been a huge supporter of OpenID from the start, if for no other reason than to enable MyVidoop users to go as many places as possible. Translation: bigger ad rates. Of the 10,005 web sites that currently support OpenID, Vidoop shares ad revenues with all of them. And Vidoop has hired the chairman of the OpenID Foundation, Scott Kveton, as its VP of open platforms.

I had dinner with Vidoop co-founder Luke Sontag last night. He was outspoken, as usual, on the virtues of OpenID, but also on how Microsoft’s bid for Yahoo makes today’s development even richer for him:

“For Vidoop, the Microsoft deal means one less potential partner that we have to explain the importance of ‘corporate grade authentication’ to,” he said. “I’ve always wondered why big companies get great access security, but the consumer companies give us no more protection than a measly password.

“A password that even Bill Gates has acknowledged is the weakest link in online security,” he said.