Infrastructure

A reader of Kevin Drum’s writes in response to Kevin’s piece on the new dotcom bubble:

See, this is the kind of crap you get when people look at "Silicon Valley" and see Facebook. The investors I have spoken to accept a fairly stable universe of Google, Facebook, Twitter, Apple, Amazon and Netflix into the mid-future. Nobody likes to look any farther down the road than two to three years anymore, so that’s the horizon.

The excitement, the "bubble" if you will, is not about creating another Facebook, it’s about finding a way to enhance and monetize an established internet ecosystem. It’s the cloud, it’s big data, it’s Hadoop, Cassandra and Mongo on commodity hardware, it’s faster, cheaper storage subsystems, it’s scalability and flexibility, the sudden newfound ability to just "switch on" five hundred new servers with no capital cost, and turn around and switch them off on Tuesday. It’s deep, low-latency analytics, clickstream analysis, social media mining, and targeted marketing. It’s mobile localized communications. It’s seamless integration between the OS, the browser, the web application server and the data, all stored in remote indexed and optimized servers that autonomously move the data closer to the user.

It’s really interesting and exciting stuff, and all the pieces are almost there. They’re still trying to figure out how to scale web properties to hundreds of millions of users, which is incredibly hard, especially because most of them were built on the previous generation’s technology — think LAMP stack with sharded MySQL and Memcached. But the infrastructure is catching up. The hardware is actually moving backwards: lower-power, cheaper, slower processors, memory and disk, but now in a clustered and distributed environment that, because failures in that kind of environment are a common event, protect themselves against failure (think multi data center replication, autonomous P2P status monitoring, disk writes to append-only tables before memory writes…).

But most people don’t know or care about that stuff. They just know about Facebook and Amazon and Apple, and they get frustrated when they don’t work. And that leads to these articles that get the core part of the conversation so dreadfully wrong.

Erik Kain

Erik writes about video games at Forbes and politics at Mother Jones. He's the editor of The League though he hasn't written much here lately. He can be found occasionally composing 140 character cultural analysis on Twitter.

24 Comments

  1. Obviously I am part of the chorus of people hoping (praying) that Google eventually creates a suite of tools so seamless that I can use them for just about everything. Right now I am looking for cloud storage that is as easy to use as a thumb drive. Amazon disappointed. Google Docs isn’t robust enough. I’ve heard good things about DroBox but if I have to register for one more website I am going to lose it.

    The new Google Music site seems to be as awesome as I hoped on first view. I deleted 3 GB of music off of my work computer because it’s all out in the ether now.

    I guess what i am getting at is that the first dot com boom was like the Wild West with crazy speculation and spectacular successes and failures. This new dot com boom is much more conservative and capital isn’t flowing like water. It’s not necessarily about completely new ideas. It’s about perfecting our experience. Shaving off the rough edges. A giant leap towards the Singularity.

    • A 3-GB thumb drive is available for about twenty bucks these days. Who needs a cloud?

      If nothing else, it’s harder for rightsholders to decide that the content server has violated some term of the contract and yank all their content (which is what happened to Netflix, and that’s why the streaming and DVD-rental services were suddenly unbundled from each other)

      • Thumbdrives are also the unprotected-sex-having intravenous- drug users of Enterprise IT. Notorious malware carriers.

          • I got a job at a very large software company in the northwest. Per the company’s security policy, Pocket PCs were banned. This was inconvenient to me as I was an avid Pocket PC user. Yet… smartphones were not banned. So I ended up getting a smartphone and life went on better than before. It gave me the perfect excuse to tell my wife, “I have to get a smartphone. For work.” (Made all the more plausible by the fact that I worked in the smartphone OS division of said software company).

          • Having a “no Flash drives or external memory” policy is a good idea, but how is it then acceptable to just go get the files from some external server?

            Of course, any company that’s serious about security isn’t going to have its production machines connected to the outside anyway.

      • I think people use clouds and thumb drives in extremely different ways. We’re talking about apples and oranges here.

      • We could use one. Our library has gotten so big we have to keep it on two separate portable drives. Backing them up on a regular basis is a royal pain in the arse.

    • I’m paying for storage on Google. Google Docs rocks for me. All my images are up on Picasa, most everyone I give a damn about is already on Google+, the only thing I use my external drives for anymore is backing up my crappy Windows environment, and the only reason I keep that around is for my few remaining braindead MSFT-centric clients.

      • Sorry Blaise, you’ve lost this argument–you’ve been very cleverly checkmated by the Duckman–big time.

        First of all, one CANNOT be a quack without having a M.D. at the end of their name–Bachmann has never represented himself as a doctor of medicine. It could be that you’re mixing him up with Marcus Welby MD, the TV doctor–perhaps that’s your favorite TV show(?) Now if you were to say, no, I didn’t mean Bachmann was a MD, I meant he was a Ph.D. Well, that’s even more troublesome–The Union Institute or the Union Graduate School, or the Union Institute and University NEVER offered a degree in Clinical Psychology.

        Blaise, you can’t win ’em all. You were also dead wrong in your remarks yesterday about Liberals and the military. You, being a professional Lefty and soldier represent a very, very small percentage of military personnel. Frankly, I’m very puzzled by your frequent berating of our military. I can’t help but think your must have been a very big supporter of John Effin Kerry. You know, the guy who served in Vietnam. The guy who didn’t have the stones to throw his own medals over the White House wall–no, he had to throw someone else’s medals over the wall. We have to thank the people of Ohio for making it impossible for this haughty, Thurston Howellish (forgive us Thurston and Lovey–we still love you!) affectatious fraud from living in the White House. I’m sure you’ll at least agree with that!

        As our beloved resident philosopher would say, Rock On!!

        • C’mon now Blaise–even you can’t be that humorless that the thought of such a “towering” intellect as yours sitting in front of a TV watching Marcus Welby M.D. is just damn funny.

          Blaise, for all of your impressive intelligence, (it is impressive) you’re sorely lacking in one of the most important of all–it’s called self-deprecating intelligence. Some have in bucketfuls. Some lack it entirely and don’t get it no matter how hard you try. I’d say you need a bit of work.

          • Specifically Blaise, you were mated when
            Density Duck asked you, “does a dog have the Buddha nature?”

            You have to give the D-Man a bit of credit for the clever, subtle, take down. Your reply left you with zero options.

          • Uh, curiously, I find dime store Zen a bit funnier than you do. It’s sorta like the Wisdom of Space Ghost.

          • Well Blaise, here’s one I guarantee you’ve never heard of and if you have and can demonstrate you understand the meaning of it, I will self-immolate myself.

            “No more above, no more below, so I leap.”

          • Blaise, just curious–how long have you had this rather tenacious case of Bush Derangement Syndrome? It must just kill you that W has a substantially higher IQ than JFKerry. And that he was a very, very good pilot of F-102s. What, the United States didn’t need an air defense? He also volunteered to serve in Vietnam but it just so happened that there was a beginning of a draw down and F-102s were being replaced with superior aircraft. You can’t just jump from one plane to another and not miss a beat.

            Did you have the skills, intelligence, and aptitude to fly F-102s?

          • Oh puh-leeze. John Kerry was a medal chaser, a glory hound, a would-be Teddy Roosevelt. Bush43 was actually a better pilot than John McCain, who crashed four, count ’em four aircraft: he’d get his scrotum all bunched up and stalled on approach. Anyone else would have been grounded. But all three, Kerry, Bush43 and McCain were what Creedence Clearwater Revival would call Fortunate Sons. Not one of them worth a bucket of warm piss.

          • Yeah, ol’ McCain, our big hero POW. So put upon, he’s out there drinking tea and giving interviews to the Cuban press at Committee for Foreign Cultural Relations, a long way from the Hanoi Hilton where he claimed he spent his time, helpfully translating his own interview into Spanish for Fidel Castro’s Granma newspaper. And making multiple PR tapes about how awful the Americans were, McCain should have been court martialed upon return to the USA for making disloyal statements and fraternizing with the enemy, but of course we didn’t do that because his Dad was an admiral. His whole POW story is so fishy, it stinks to high heaven.

          • BP, “Not one of them worth a bucket of piss.”

            So that’s what you have to say about a man, a Navy aviator, your fellow comrade, soldier, a POW for 6 torturous years? My God, you are a hard-hearted, bloodless man incapable of any sense of empathy. Who are you trying to impress with such caustic, deeply hurtful remarks? You’re a pathetic Alpha wannabe with your own mediocrity painfully staring you in the face. Get used to it, cause that’s all there is–your heartbeat becoming weaker with every breath. Is this your idea of badass humor? You fail miserably, sir. I think we all know who is really worth a “bucket of warm piss.”

          • Aw, spare me. I have seen sailors, prisoners of war, too. Hard hard hard men. Men who never made PR tapes for the enemy like that asshole John McCain and don’t you say he was tortured to make them. I say these things because I mean them. It’s all a question of mind over matter. I don’t mind, you don’t matter.

            The fainting couch is over that-a-way. You will find no sympathy over here, If only I could bounce a box of Kleenex off your head, the scene would be complete and I should die a happy man.

  2. I have two points to make on the broader subject.

    First, “cloud computing” has been “right around the corner” ever since I was in college. Doubtless, long before that. But it hasn’t happened. It sure seems closer to happening now as everything becomes more meta-virtual, but while they’re saying that no one wants to think more than 2-3 years into the future, I would say that we’re twice that far away before it becomes really commonplace.

    One of the reasons – and where I thought you were going to go with this with the “infrastructure” title – is the lack of thorough Internet structure. Our Internet infrastructure is too weak. It’s insufficiently reliable. Silicon Valley can work around this with better offline features, but while I can access information on a server in North Carolina from anywhere I have an Internet connection, I can access information on my local laptop from anywhere I have that laptop. We’ve got Wifi and 3G/4G, but none of it is yet absolutely universal.

    That is the real roadblock. As someone who has an unbelievable number of computers sitting around and who goes back and forth between desktop, laptop A, laptop B, and so on, I look forward to it. But as yet, I simply can’t trust it.

      • This is even more true outside the US. All of New Zealand and Australia’s internet connection to the rest of the world comes across a cable with less than 1 Terabit / second capacity. Any service that requires me to exchange a lot of data on the internet to function is a non-starter in these parts.

Comments are closed.