Articles about computer software, hardware, and the internet.

Off To MinneDemo!

Its MinneDemo 3 time... Minnesota's version of DemoCamp... which is a spin-off of the BarCamp unconference. Apparently the Minneapolis BarCamp had more attendees than any other un-conference this year (besides one in India).

I just checked the wiki containing RSVPs... free beers for the first 200 attendees! Also, it appears that Little Bobby Tables will be joining us:

He's a joker, that Bobby Tables...

Even Newer Buzzwords: Network CENTRIC Computing

Well, no more than one day after I blogged about it, Pete Lacey has three new buzzwords:

  • Network-centric computing (NCC)
  • Network-independent computing (NIC)
  • Network-centric governance (NCG)

The air is getting a little think now... I'm not sure how this helps the debate, it seems a little complex. He also lumps SOAs into the category of architectures that try to abstract away the network. I'd argue that only a poorly written SOA would attempt that. EJBs tried and failed at that many a time, then wrapped themselves in WSDLs and rebranded themselves as "service-oriented architectures."

Bullshit. EJBs suck. Abstracting away the network is stupid. The backlash is a major reason why EJB 3.0 supports Plain-Old-Java-Objects for persistence.

I also disagree with his blanket grouping of CORBA, DCOM, and EJBs as SOAs. That's clearly an attempt to group 3 known losers with something he doesn't like (Service-Oriented Architecture) as opposed to his pet project (Resource-Oriented Architecture)... I'd argue that the 3 losers' primary abstraction is an object... So you tell me, is an object more similar to a resource, or more similar to a service?

Hopefully this buzzword bingo will shake out, and we'll finally be able to architecturally categorize the 3 losers, as well as SOA/SOAP and ROA/ReST.

New Buzzword: Network Oriented Computing

I've frequently criticized ReST on my blog... in the problem spaces I deal with, it's too narrow of a pipe. Being resource-oriented is fine for some situations... but almost every single one of them violates the Don't Make Me Think! principle... so I hesitate. It may be my background, but I always preferred thinking about problems as services. Even if SOAP goes the way of the dodo, people who think in terms of services instead of resources will always write more interoperable code... because they're concerned with connections between resources, and not the resources themselves.

Recently, Peter Lacey suggested that a lot of this animosity may be due to improper terminology, and offered some structure. UPDATE: he just added three new buzzwords. Joy. Anyway:

No matter which definition works for you, though, SOA is misnamed. So I’ll leave you with some updates to your lexicon:

  • Network Oriented Computing (NOC): An approach to computing that makes business logic available over the network in a standardized and interoperable manner.
    • Service Oriented Architecture (SOA): A technical approach to NOC that has a non-uniform service interface as its principle abstraction. Today, SOAP/WS-* is the chief implementation approach.
    • Resource Oriented Architecture (ROA): A technical approach to NOC that has an addressable, stateful resource as its principle abstraction. Today, REST/HTTP is the chief implementation approach.
  • Business Service Architecture (BSA): An unnecessary term (also not an architecture) that tries to make the obvious something special. Aka, business analysis. Aka, requirements gathering.

I like the distinction... I especially like how he separates BSA out of the equation completely. However, I think NOC is missing a few (older) subsets: CORBA, DCOM, and EJBs in particular. He mentioned these on his blog, and it seemed that they had the same goal as being an NOC. In fact, all 3 would probably label themselves NOCs... but they all failed to deliver on promises.

Why did they fail? Because they relied too heavily on a one platform or language? Not really, CORBA was cross platform and cross language... Did they fail because they were too complex? Perhaps... or did they fail because they were too object-oriented in their design? I'd argue it's all three... nevertheless, these failed attempts have more in common with Resource-Oriented Architecture than Service-Oriented Architecture.

Currently, SOAP claims to be platform and language neutral... but the WS-* stack is so complex, interoperability between SOAP toolkits is still a ways away... Microsoft did this on purpose to enforce vendor lock-in, and nobody seemed to complain. ReST folks assert that their stuff is truly easy and cross-platform... but that's because Microsoft hasn't tried to ruin it yet. Just wait until they try to cram WS-* security and validation into ReST... then you'll know sorrow.

But stepping back from implementation details, I just don't think an ROA is the best long-term arrpoach. These days, I'm much more into the way Freebase does it... here's a look at their request structure:

{
  "query":[{
    "id":"/common/topic",
    "properties":[{}],
    "type":"/type/type"
  }]
}

Beautiful, simple, service oriented but with the clarity of ReST.

Protocols like this will be the wave of the future, IMHO. No need to remember a bazillion resources because you only have four verbs... no need for the ungodly complex layers on SOAP... just pure data messaging. Plus, you can embed JavaScript code in there to make it as extensible as any Turing Complete language. All you need is a JavaScript interpreter for your language... like Rhino in Java.

I'm sure my JavaScript suggestion made Ruby/Python/Perl folks cringe... but deal with it. Its there, its popular, people know how to use it, it has regular expressions, its good enough.

Greg The Architect

I just came across this oh-so-nerdy site called Greg The Architect from Todd Biske's blog. Here's the latest episode:

http://www.youtube.com/watch?v=CnhEfxxhg34

I though it quite fitting since Gartner just came out with their 2007 Enterprise Content Management Mystic Grid. Seriously boys... Open Text as most visionary? Somebody was shaking the magic 8-ball fast and furious this year...

Dynamo, Hadoop, Memcached, And Groundhog Day!

Apparently, just like Google, Amazon realized that silly things like transactions don't scale, neither do auto-incrementing counters, nor table joins... so they made some kind of wacky database called Amazon Dynamo that doesn't require them. The entire system is essentially blobs of binary data, accessible by unique keys. Each key is mapped to a unique URL, which makes the ReST fanboys drool... its fast and highly available, but it doesn't do a majority of what a typical database does.

In other words, they've replaced their database with a freakin' file system! Or, more precisely, its a highly distributable cache that is freaky fast, freaky scalable, and ignores silly things like security:

Other Assumptions: Dynamo is used only by Amazon’s internal services. Its operation environment is assumed to be non-hostile and there are no security related requirements such as authentication and authorization.

erm... that's a pretty big assumption! Hopefully they don't put credit card numbers in that thing...

They aren't alone... many of the biggest sites on the web use a similar technology, called memcached. There are other radical database replacements out there, such as Hadoop, and Mnesia, both are Sam Ruby's Long Bets... and who knows what exactly the guys over at Freebase are doing behind the scenes... but the general feeling is that fast databases in 3 years will look nothing like they do today.

I just keep getting this weird feeling that we've been here before... Its almost like people have given up on the idea that you can ever truly capture data.

For example: what's on the internet right... NOW? OK, how about... NOW? How about NOW? Firstly, its an unanswerable question... and even if we could know it, things like proper ACID transactions certainly would never help. By the time the transaction finished, the data's no longer valid.

As organizations place more data on easily accessed internal systems, the question turns to "what' on MY network right... NOW?" At present, we can answer this. But what about the future? Financial and government people will demand all kinds of horrendously complex solutions to ensure zero transaction failures and 100% accurate data for auditors... but that doesn't stop one clumsy employee from typing the wrong number. Isn't there some wasted energy there?

Bruce Schneier says a good security plan has three sections: Protection, Detection, and Reaction. The math is pure, but the weak link is the human operator. On very large and very complex systems, failure is the norm, not the exception. You can't focus all your energies on protection from failure, because then you'll never develop the tools or skills needed to find and fix inevitable problems.

The same may hold true for databases... for sufficiently complex systems, a database only represents a vague approximation of the state of your organization. This problem will only get worse when people implement SOA across the enterprise... which will enable complex interactions and dependencies across business units that were never before possible... Empowering? Yes. Important? Yes. Scary? Yes.

As bandwidth gets cheaper, many people will be testing the truth of that statement... But, since fortune favors the prepared mind, I think we're safe ;-)

Stupid Corporation Tricks

A few items of interest...

1) AT&T sure loves freedom of speech... that's why their terms of service explicitly forbid you to use AT&T's network to criticize AT&T. That kind of puts a kink in their customer service process, doesn't it?

"I'm sorry, we can't help you fix your problem. Even though you're calling from a MCI pay phone, our network is still AT&T... and if you say anything is wrong with your services, that counts as criticism, and I have to hang up on you. How about if I give you my Skype ID, and we'll go from there? Don't worry, this will only be until we switch our internal systems over to Sprint."

2) Need another reason to not offshore your software projects? Besides the fact that junior software engineers in India make $45,000? Apparently IBM is patenting offshoring... thus, you can't send any software work outside the US unless you shell out some cash to IBM.

A bet a few out-of-work IT guys wished they had thought of that... Forget pragmatic programming and cheese moving, I'm going to patent layoffs! Then nobody can fire anybody unless they pay me a royalty.

My rates are reasonable.

UPDATE: Apparently IBM has withdrawn their outsourcing patent application. I can only guess that this entire fracas was due to a typo somewhere... IBM meant to file the patent in Crazy Town, but due to a clerical error it wound up at the US patent office.

Boy, is their face red!

ANOTHER UPDATE: AT&T is now claiming it's not out to silence its critics, rather those obtuse words in their terms of service are to stop child pornographers... riiiiight... cuz you need to treat ALL your customers like criminals to catch them. And for some reason, this goober believes AT&T.

I'm reminded of something Alec once told me:

Whenever someone says "won't someone think of the children?!" I put one hand on my wallet, and the other on my civil liberties... because odds are "someone" wants them both.

a t & t 2 a t...

Spam, Identity, And Scary Frickin' Clowns

What is identity? Is your identity what you say you are? or is it what others say you are?

On the web, you could be a spammer or a scammer... I don't care what you say you are, I care what others say about you. Just like your Google PageRank, if other link to you and say you're OK, then you're probably OK. Not OK enough to trust, but OK enough for me to find you first...

But who does this external validation of your identity? Who gets the power to say who you are? Your identity can't be purely what somebody else says... otherwise you're trapped in a Kafka-esque world like Bingo The Clown-O.

http://www.youtube.com/watch?v=TzeD0ibhkHA

Perhaps the more vocal one is, the more their identity should depend on the opinions of others... whereas the meek and quiet should have greater power to say who they are...

In any event your identity is too important... it cannot be trusted with any one person, one organization, or one security protocol. You need multiples of each, always competing to find bigger, better solutions. You need a web of trust so people can get a fuzzy idea of who you are... precision is impossible in real life, so why do so many think its possible on the web?

Ah, I miss the good old days with PGP key signing parties...

Take My Privacy, Please!

Now that school's back in session, many students are being advised to not place embarassing photos of themselves online... employers these days are making it a habit of Googling potential hires... and there are numerous cases where people have been fired because of "unprofessional" content on Flickr, their blog, or a Facebook profile.

My advice? Any company that fires kids for acting like kids is a horrible place to work... Plus: you don't need them half as much as they need you.

Hear me now: we are looking at a bit of a demographic crunch in the next few years... there are shockingly few talented people able to fill needed jobs. A junior software engineer in India can demand $45,000 for their first year... so wave bye-bye to cheap offshore talent. When even China is having labor shortages, something is very wrong. The Economist calls this the global talent shortage, and it's gonna get messy.

Employers: for your own sake, understand that your employees are people, not profiles. Yes, the latest generation has done some crazy stuff... but it's probably a lot less crazy than what George W. Bush did in his youth... the only difference is the openness. The older generations think the young must be lunatics for posting their exploits for all to see... youthful indiscretions are to be supressed!

Frankly, these kids don't understand what the fuss is about...

Recent college graduates have never known a world without reality TV. Dig this: The Real World has been on MTV for fifteen years. Graduates have seen people they admire make complete idiots of themselves, but all can be forgiven if you're a decent person.

I'd wager the following will happen:

  • Unenlightened and dull organizations will implement policies against hiring people with embarrassing profiles.
  • Exciting and fun companies won't care, as long as you're a talented and genuine person.
  • Dull companies will hire boring, untalented, and duplicitous employees.
  • Fun companies will hire exciting, talented, and honest employees.

Take a guess who will win in the end.

Now... there are some situations where the tendency to reveal too much will get you in hot water. Legal, government, the secret formula for Coke... although many people believe that more openness even in those areas will help the world in general.

Pragmatically speaking, there are certain roles which (at present) necessitate a bit of secrecy... and a blogging maniac would be ill-suited for them. However, that doesn't mean that it should be a general policy for an entire organization! That's shooting yourself in the foot.

If I ran a mutual fund, I'd take note of any company with a rigid policy of firing employees based on bad Facebook profiles... because they will almost certainly fail to hire enough new talent in the oncoming talent crunch.

Some Radical Ideas About Future Web Apps

Still jet lagged, I hopped online at 5am today to check out what I've missed on the blogs... and I came across two articles that together painted an interesting picture of the future of social software sites and web apps in general...

As I've mentioned before, social sites like Facebook are a house of cards. I never really saw the draw of going to a walled garden to visit my friends... once people can easily migrate their contacts, these sites lose most of the power to captivate their audience.

Jon Udell beat me to this realization a while back... with these pithy word from his friend Gary McGraw.

People keep asking me to join the LinkedIn network, but I’m already part of a network, it’s called the Internet.

Oh, snap! Harsh, but completely true... Why should a person need to have accounts of 5 Instant Message servers, 5 social sites, and 5 presence sites to keep in track of contacts? Can't we make it easier?

Now, I don't believe open standards will ever get us there. Nope. Nuh-uh. No way. We already had a perfect open standard for IM interopability -- Jabber -- and when Google launched their Jabber-based GTalk service, they turned off the compatibility layer. Grrr... I have a well-documented dislike of 90% of the standards in existance... suffice it to say, if you build it, they won't come.

What WILL get us there is one of the crazy startups that Joel Spolsky briefly alluded to at the bottom of his Strategy Letter VI. Precompiled libraries which yield consistent methods for doing everything a good web-app should, but entirely in browser-side code, like JavaScript or Flash, but light years ahead of current Rich Internet Apps.

This layer obviously handles all the standard UI widgets so you never need care about browser compatibility... but it would also support rock star word processors, spreadsheets, inter-process communication, instant messaging, cut-and-paste between separate AJAX apps, as well as multiple single-sign-on security protocols.

Sharing data between two web apps built on this framework will be a breeze.

Of course, this would be in a 300Meg download... and initially it will be scoffed at. In the same way that people are currently mocking Adobe AIR for being so heavy... but the world is changing, and bandwidth is getting much much cheaper.

Joel believes that people shouldn't care about bandwidth or performance: just make cool stuff and the hardware will eventually catch up. Ummm... Note to Joel: this principle obviously did not hold true for Java Swing applets... which were a 300 Meg download 5 years ago, ungodly slow, and are now "dead, dead, dead." So use caution with such blanket statements...

Anyway, does this mean that existing internet apps will switch over to the new rich-internet APIs for easy interoperability? Almost certainly not, but who cares? Facebook and GTalk will likely stay the same... but the latest "cool" sites will be using the latest "cool" technology. And quite bluntly, Facebook's cools factor declines as the average user's age increases. And besides, the next generation of web APIs will let you re-create Facebook in a few months...

By the time rich internet apps reach critical mass (2-3 years), the younger generation will believe Facebook is for dinosaurs. They won't care one bit about connecting with those on legacy systems... so who cares if Facebook doesn't integrate? And, in the highly likely scenario that somebody wants interoperability, any developer worth his salt should be able to write a spider to extract data from a web app, and wrap the next-generation APIs around it.

This is just the next step in the constant forward-march of technology: make information easy to reuse, tell people about it, and innovation will follow. This is currently dubbed Web 2.0, Enterprise 2.0, and a handful of other things... but these software principles are no different today than they were 30 years ago.

In short: nuts to standards; long live disruption.

How To Boost Your Google Rank

I get asked this question on occasion, and people are usually surprised how little page rank has to do with technology.

There are a lot of salesmen on the web -- called Search Engine Optimizers (SEOs) -- that claim that they can improve your rank with a bunch of META tags, blogs, and analysis. Some of what they say is common sense... its probably just a rehash of Google's own advice for improving your rank.

Everything else is a con.

Based on what I've read and tested, I think the following four points are all you need to know to legitimately boost your page rank:

  1. Have useful content that Google can spider
  2. Offer a useful service
  3. Use public relations and marketing to get high-quality web sites to link to you
  4. Avoid a rigid thesaurus, or a rigid style guide

The first one is obvious: make useful content that people want. Making spiderable pages is the only technical requirement here, and is pretty simple. Put your content in HTML, XHTML, plain text, or PDF. Do not require JavaScript or Flash for people to see your content, or link between pages: use it only to enhance content. Simple. Done.

The second is more important if you do not have a lot of content on your site. Its okay to have a site that is all AJAX, as long as it provides a useful service that lots of people are willing to talk about. However, doing this is more difficult...

Which brings us to the third less obvious tip... people have a hard time understanding that the Google rank is very similar to your identity or reputation. Its not about what you say, its about what other people say about you. If a big, important site links to you, this usually means you have something useful on your site. This trick is essential for keeping spam sites (or splogs) low in the rankings.

If you want high rankings, you need to schmooze those folks who run the big important sites to write an article about you. That's PR, and I'm not qualified to talk about that... suffice it to say, it's no longer a technical problem.

The fourth tip is a bit painful, and not often mentioned, but its true. I've read several books on information architecture for web sites. Some of them are useful, but unfortunately they lead people astray when it comes to Google rank.

Most communication experts believe sites should have a consistent vocabulary and style. The obvious benefit is that its easier for a human to read content if its a consistent style. It also gives the illusion of a single author, as opposed to a committee.

However, consistency has a huge down side when it comes to the web. People searching for your site don't care one bit about your vocabulary! They have their own vocabulary, and search with their own terms. Google won't do synonyms, so if their exact word is not in content on your site, you will not be ranked. Simple as that. Site indexes with synonyms do help, but might not be enough.

For example, take my post about how I hate String Theory. Its one of my site's more popular pages. When people search for string theory wrong, I'm ranked #55 . When they search for string theory crap my site is #6.

However, if they search for string theory sucks, my site is nowhere in the 763,000 results. Nowhere! Those poor souls who hate string theory as much as I do are missing out on my rant! This must not continue...

As an experiment I made a modification to the original article on Thursday, August 31st, 2006. I added the word sucks very near the phrase string theory, then played the waiting game... As of September 04, if you search for string theory sucks, my site is #7.

Of course, after I publish this article I'm sure my rank will increase even more. After all this Google inspired navel-gazing, my site is lousy with the phrase string theory sucks.

In conclusion, put down the Search Engine Optimization for Dummies book, and dust off your copy of How to Win Friends and Influence People. Your Google rank is much more about PR than technology. Maybe an intranet-based Google appliance cares about those META tags, but nobody else does.

UPDATE 9-04-2007: A full year later, I checked my Google rank for string theory sucks, and my site is #2 and #3. For string theory crap its #1. However, for string theory wrong it only rose to #39... probably because of a book with a similar title.

Google Owes Me A Pony

I was always fascinated by economics... but every time I tried to study it I was put off by the sheer idiocy of it all. Pretty much every founding principle of economic theory is complete and utter crap, and it infuriated me that educated people bought into it. Economists totally tarnish the nobility of the Nobel Prize...

That was before I picked up the amazing book The Origin of Wealth, which is about a new field called complexity economic theory. Complexity theory states pretty clearly that "traditional" economic theory is crap... and replaces it with a vastly more enlightened model. The economy cannot be modeled with automatons with perfect knowledge buying widgets; its based on the principles of complex, dynamic, evolutionary systems... and has many emergent properties that are impossible to predict, although clearly not random. Its pretty academic, but if you have a basic understanding of physics, computer modeling, and evolutionary biology, its a must read.

Anyway, In the book the author tries to answer the only economic question worth asking:

  • How is wealth created?

Traditional economic theory states that by specializing in a trade -- or a product -- you can create wealth... I make axes, you make clay pots, we trade, and wealth is created... but why? What is this thing we call "wealth," and how exactly does trading and commerce create it?

We should emphasize here that the economy is not a zero-sum game... Many used to believe that there is a fixed amount of wealth in a country, determined by the natural resources a country has, its labor, and its ability to turn the two into wealth through technology. Nope. The economy is not a big pile of gold. People shouldn't have to lie, cheat, or steal to get their share... That philosophy is called mercantilism (thanks solistics), and is rejected even by traditional economists... although tradition economists could never get the math to work out to explain how wealth was actually created...

To illustrate further... On occasion, a king, a country, or an organization stops innovating, and simply coasts on the capital from previous glories. At this point, the local economy does resemble a zero-sum game, and no economic progress is possible. In fact, if the competition (ie, other countries) continues to innovate, the organization will get poorer. Throughout history, closed societies (North Korea, Fundamentalist Muslim nations, Ancient Sparta) failed economically, whereas their "open" counterparts (South Korea, Dubai, Ancient Athens) became economic powerhouses.

Throughout history, zero-sum thinking has caused wealth to stagnate... whereas sharing, cooperation, and innovation always generated new wealth. By following win-win situations, new wealth can be created almost by definition... But how on earth does it work? What could explain it all?

Wealth Is "Fit Order"

The author put forward a pretty cool theory: wealth is fit order. By this, he means that wealth has little to do with natural resources, capital, or labor... its not even so much about technology. Instead, wealth and knowledge are fundamentally the same thing!

Note: raw data is insufficient to create wealth... This difference is partially semantic, but it's important. Knowledge needs to be transferable and useful to others. Thus your religious faith and supernatural beliefs have personal value to you, but no economic value... In fact, they can have serious negative value if you listen to shady preachers or TV psychics...

True wealth only occurs when a piece of physical technology and a piece of social technology mix into a "business plan." In addition, implementing the plan needs to create higher levels of order from available raw materials... pedantically speaking, a plan must be thermodynamically irreversible, decrease local amounts of entropy, and thus consume lots of energy. I have my doubts if this is a requirement -- deleting embarrassing emails and demolishing buildings can alone increase wealth -- but for now I'll agree.

In the next step, the information -- or "order" -- is tested out in the "economy" to see if it is "fit". Beinhocker provided a wealth (har!) of data demonstrating that the economy is an evolutionary system, which means it adheres to a basic evolutionary algorithm:


10 differentiate()
20 replicate()
30 selectForFitness()
40 GOTO 10

This is survival of the fittest. Most ideas are slight variations on existing ones, and on occasion you see radical new ones. The radical ideas are more likely to fail, but every once in a while they completely change the economic landscape.

If the business plan is copied by others, it replicates. Now, sometimes the business world is trendy, and people copy ideas for no good reason... but if the idea lasts over a few years, it can be considered fit. Fitness, naturally, is dependent on your environment: a polar bear is a rock star on the ice, but wouldn't survive long in the Sahara...

Fit order creates wealth.

Skeptical? Consider this: did the invention of marketing create wealth? How about public relations? What about customer service, or human resource departments? Do any of these things produce physical objects of material value? NO! But do they produce wealth? YES!

How do we know? Well, corporations that adopted good practices in these "service" areas outperformed their competitors. They got better employees, better customers, better products, and better market capitalization. In the dog-eat-dog world of capital markets, companies with these services attracted and generated more capital. If additional wealth wasn't generated by these activities, no corporation would implement them for long.

Clearly, service industries generate wealth just as agriculture and manufacturing. Simple as that!

All of these are examples of fit order. Marketing is a good way to make a bad piece of technology sell better, or to help inform people about good technology. Public relations keep you out of trouble when the news is bad, and helps you toot your horn when the news is good. Customer service ensures your customers are happy, and willing to pay a premium for your product. Finally, good human resources ensure you locate and retain the best employees.

The job of a computer geek is even more abstract... we string together ones and zeros into applications that make our customers feel stupid, then we mock them. By some bizarre chain of events, the economy appears to prize this career over the creation of food... Similarly, school teachers are vital for our next generation, but they don't make nearly as much money as sports stars... Adam Smith called this the diamond/water paradox: water is essential to human life but is nearly free, whereas diamonds have little to no value and yet cost a lot... Price is all about supply and demand, and has little to no bearing on the actual value of something.

Ummmmm... So Why Does Google Owe You Anything?

If Beinhocker is correct, and there's a direct correlation between wealth and knowledge, then public data stores like Google are modern wealth-creation engines... This idea in general was covered by The Economist and Jimmy Guterman at O'Reilly recently, but I don't think either understood the full implications...

When the modern banking industry began, very few people saw it as the wealth-creation mechanism that it was. Many saw it as a wealth protection and re-distribution system. Investors saw them as a "safe" place to put their money. Bank managers understood profit, loss, and risk... which meant charging the right interest in order to make a profit for the bank and its investors... however, many still regarded the economy as a zero-sum game.

Since Google stores so much information, they can mine it to find new kinds of ideas, new concepts, and create new knowledge. Thus, they can sift through the cruft and chaos on the web to find the order. Since Google tracks everything, they can watch trends, and monitor the popularity of certain ideas. Thus, they have their fingers directly on the pulse of what kind of order is fit.

If wealth is actually "fit order," then giving information to Google is like putting money in a bank, but not getting interest! That's why Google owes me money... or at the very least they owe me a few good ideas. However, I already have more ideas than I need, so I'll settle for a pony.

The Economist compared Google to some of the original banking families... not because they do the same things, but because they both wield tremendous power. Well, Google indeed holds the same power... not by coincidence, but because Google is creating wealth in the same way the Rothschilds and the Warburgs did. It by no means a coincidence that they wield the same power: if Google was not instrumental in the creation of wealth, they would be on the ash-heap of history by now.

An older article in The Economist asks how a company like Google can be so big, and yet claim to not be motivated by money. I believe the answer is simple: Google isn't motivated by cash; Google is motivated by wealth. Cash is only one mundane variation of wealth, which becomes less and less useful every year. What use is your gold in a world with Star-Trek replicators? It isn't worth nearly as much as your ideas are... and desktop replicators are closer to reality than you may know...

As such, Google might just understand things about the new economy that others fail to grasp... or perhaps they just stumbled ass-backwards into one of the greatest wealth generation systems since J.P Morgan.

Either way, good for them.

And gimme my pony.

Forget Wikis, Long Live Freebase!

If you haven't yet heard of Freebase, you will soon... It takes the concept of a wiki, and expands upon it for structured content, including metadata, and content types. Its like Wikipedia for databases.

The Freebase tutorial has a pretty cool overview of how it works, and its API.

Each content type -- such as person, or city -- has associated metadata -- such as birth date, or population. Instead of having to scan wikipedia to glean this info, its in a structured yet free-form database. This means its a breeze to get a report of cities by population... or you can do joins on data to get reports with richer data. Say, a clever table join to determine a film star's Kevin Bacon rank...

Its still in alpha for the moment. Currently, only those selected as alpha testers get to add or update entries -- like me, suckas! -- however, they recently opened it up to allow the public to access the existing content... provided you have a login. If anybody wants an invite, I got a few to share...

I'm fascinated with their mashup enabled API... Its entirely based on JSON web services! That means you can query the database using just JSON formatted data, and you'll get back a JSON response.

drool... its very close to the fusion of SOAP and ReST that I've been too lazy to invent myself ;-)

If that weren't enough, they also use a JavaScript based templating language for their entire UI! Its called MJT, pronounced "midget." No PHP, no Ruby on Rails, no J2EE portal server needed... just pure JavaScript! A relatively savvy web developer can write an entire RSS feed reader in under 100 lines of MJT code... and no, I'm not being an optimist.

A minimal sample looks like this:

<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8"></meta>
<script type="text/javascript" src="http://mjtemplate.org/dist/mjt-0.5/mjt.js"></script>
<title>a minimal mjt example</title>
</head>
<body onload="mjt.run()" style="display:none">
    running mjt version $mjt.VERSION
</body></html>

Put MJT and JSON web services together, and it means you can write robust database applications in pure Javascript! This means those enterprise mashups I swoon over will be even easier to make...

I'm very curious about the back end application. Freebase allows you to create new content types in your own "domain", and "blessed" ones are available to all users... but allowing people to create their own content types with structure is like letting them create their own database tables... and joins between such free-form tables usually have terrible performance. Also, object-based relational databases have historically failed to deliver performance superior to SQL.

They claim to have a radical new way of writing databases, and they're leveraging that to allow such free-form editing. I wonder if it's using Erlang/OTP, or something similar? We'll see how well it can scale to Wikipedia's size...

This things is going to be huge... and if they can deliver on their performance promises, I can see massive value in a Freebase Appliance for business intelligence or data mining.

Pure awesome...

Open Social Networks: LiveJournal Leads The Charge -- Off A Cliff

Apparently Brad Fitzpatrick, creator of the Live Journal blogging toolset, has published a "mini manifesto" about the need for open social networks. I'd suggest he get in touch with Armand du Pless, and check out Identitude... its a good first-step to breaking down the walled garden at Facebook.

Its a great idea that many bloggers have mentioned before... but there are subtle problems with it...

Personally, I'm sick of social networks trying to be all things to all people, and being greedy with their information. MySpace and Facebook are still mainly for kids... although Facebook is trying hard to reinvent itself to include grown-ups as well. This strategy will haunt them, because once kids see their crazy uncle Sammy on what's supposed to be their social site, Facebook will cease to be cool.

Ugh... seeing your parents on Facebook is like bumping into them at the mall... Imagine a tragically hip mother uttering to her teen daughter "I use Facebook too!" Dude... nails on a chalkboard. Why has Facebook ignored the #1 rule: responsible adults are frigging kryptonite to anything cool. The more Facebook tries to be "grown up" and go after LinkedIn's turf, the less cool they get... and the more opportunity there is for a startup to steal the younger crowd from Facebook.

The true solution is a network of identity providers that allow people to have one login ID, and be able to move contacts, friends, notes, and bookmarks from one site to another... then as your interests -- or age -- changes, you can move all that data effortlessly to another place.

In practice, your kids first use a tween-focused social site, then a teen-focused site, then a college-focused site, then who knows? Probably ones specific to their industry, or ones focussed on a specific hobby. The mighty mighty interweb demands diversity...

But there's a big big problem... At the latest Black Hat hacker convention, those naughty miscreants exposed security holes in OpenID and similar systems. These aren't minor loopholes, they are ways in which federated identity is fundamentally a security hole on the greater internet. Worse than cross-site scripting in some cases. Not to mention a privacy annoyance... the structure of it means your identity provider knows every single site you log in to... Do you really want Microsoft to know every site that you access?

Personally, I don't like that... To contrast, my state government issues me a driver's license, and I use that mostly to prove my age at bars -- I rarely drive. So, does the DMV have rights to get a report of every bar I frequent? Hell no... so why on earth would people be OK with their identity provider knowing every site they access?

That's one reason why I'm in favor of user-centric, distributed, identity management. Something like SXIP perhaps. See the infamous Identity 2.0 talk for an overview of user-centric identity.

I'd like to see open social networks... but if they don't nail the security problem so that it makes sense to users, and it there's a strong incentive to adopt, the effort will be utterly pointless.

(Hat Tips: Mashable, Sam Ruby, and Jeff Bohren)

Who Owns A Relationship?

After my claim that decent identity management would destroy Facebook last week, I began thinking about identity management a lot.

A whole lot...

More than is healthy, really...

I'm obsessing about the question: who owns a relationship? The answer seems obvious, but there are a lot of gotchas that make me nervous... I have to blog this before I can wrap my head around the true market value of an OpenID/XACML integration...

Lets assume I'm a salesman working for Oracle (I'm not)... and I have a friend in charge of software purchases at Boeing (I don't). Lets further assume that we set up some system with OpenID that stores that relationship data on the greater internet -- something like a next-generation LinkedIn or Facebook that "gets" identity management. Or perhaps Facebook with some OpenID magic using Identitude...

Anyway, lets further assume that the relationship is important somehow to my boss... so its managed internally by both employers, using some variation of SAML/XACML for authorization and trust inside the firewall.

Now... who owns that relationship? Um... Arguably, the humans do. The relationship is mine, and my friend's. Until a falling out of some sort, we're on each others' buddy list. My relationship, my data.

But wait a minute... doesn't Oracle have some claim to this information? Its not just a me/him relationship, I'm also a proxy for an Oracle/Boeing relationship... If I quit, they need to know who my contact is at Boeing so they can continue with the potential client/partner/whatever relationship.

Perhaps... but what if I quit and joined IBM or Microsoft? Sales people are hired for two reasons: their salesmanship, and their contacts. You can bet that they would want to know about my friend at Boeing... Likewise, my contacts were probably a major reason why Oracle initially hired me... They sure didn't mind when I left my previous employer with that relationship in tact... although they sure would like to retain control of that relationship after I leave.

Everybody wants it both ways...

If I were clever, I'd probably keep my business contacts outside of my employer's clutches... like Highrise, or at least paper copies at home. If my employer were clever, they would require me to keep it inside a strict, federated, SAML/XACML secured repository for customer relationship management, and private networks similar to LinkedIn... they'd like to make it as tough as possible for me to move my data out of their repository, even though I have a right to that data as well.

This harks back to the age-old question of "what is property," answered by one of the fathers of American capitalism, John Locke:

every individual man owns his own person; this is something that nobody else has any right to. The labour of his body and the work of his hands, we may say, are strictly his. So when he takes something from the state that nature has provided and left it in, he mixes his labour with it, thus joining to it something that is his own; and in that way he makes it his property. He has removed the item from the common state that nature has placed it in, and through this labour the item has had annexed to it something that excludes the common right of other men: for this labour is unquestionably the property of the labourer, so no other man can have a right to anything the labour is joined to - at least where there is enough, and as good, left in common for others. -- Second Treatise of Government, John Locke, Ch 5, Section 27

Despite 200 years, Web 1.0, and Web 2.0, this argument still makes the most sense... the information that I have a relationship is free to all who bother to find out... however, the ownership of the structured data is the property of whomever mixes their labor with the information. Complexity Economic Theory would dictate that's when data turns into knowledge, and therefore economic wealth.

Locke also said:

Where there is no property there is no injustice

but let's put down the book and step away from the hippie...

Essentially, I have no right to expect easy transference of data outside of my employer's clutches. Even if I typed in everything to their system, they were paying my salary at the time. Likewise, if I entered that information into Highrise on my own time, they have no right to expect an easy copy of the data into their systems.

Thus, there's actually a very strong disincentive towards making relationship data too easy to move from the internet to the intranet... Why should I let my employer get my labor and wealth (entering contact info to Facebook) for free? Likewise, why should they give me a great parting gift when I quit? To a lesser degree, this same disincentive even exists for sharing bookmarks and tag clouds across the firewall...

An OpenID/XACML integration sounds cool... but because of a lack of incentives -- and many disincentives -- I just don't see its value in today's world. At best you could import things like name and email address, asking for anything else will probably cause problems...

Something else needs to change first before that kind of identity management could find a niche...

Use Twitter: Save The World

Apparently, Twitter is now being used amongst first-responders and fire departments, for updates on what disasters they are currently dealing with.

The article gives some interesting tidbits about how Twitter played a substantial role in dealing with recent fires. Check the Los Angeles Fire Department Twitter page for your own updates, if curious...

Now, if only Ruby, Rails, and Twitter were robust enough to deal with huge volumes of emergency data. Frankly, it might be time to re-write Twitter in Erlang. Erlang is fault-tolerant, high-performance, lightweight, scripted, highly specialized for communication protocols, and its oh so weird...

As Software As A Service (SAAS) and Service-Oriented Architectures (SOAs) become more common, I can totally see a market for a robust Erlang-based web framework. Distributed scalability and performance on commodity hardware, no app server needed...

mmm... erlang on rails... Erls? Maybe OTP has some extra magic to add...

UPDATE: Hold the phone... it appears that Stackless Python is faster than Erlang for some kinds of message processing... Plus it's Python... BONUS! Maybe Twitter should replace Rails with some kind of Stackless Django...

Facebook Adds OpenID?

Well... THAT was unexpected...

A rogue hacker named Armand Du Plessis just created Identitude... a prototype OpenID system for Facebook, written in the Facebook API.

Ten bucks says he either gets shut down, or the integration won't allow you to easily extract relationship data...

Good Developers Versus Bad Developers

First, a brief grounding in the myths surrounding programmer productivity:

Myth #1: developers are like cogs in a great process machine, and can be switched out based on experience and seniority... this is fostered by those who believe strongly in top-down outsourcing, offshoring, and that the man month is a good measure of developer produtivity.

Myth #2: one uber programmer can do the job of ten programmers... this is fostered by some alpha geeks in the open source community, a handful of well paid consultants, certain Slashtards, and Joel Spolsky.

As usual, the truth is somewhere in the middle... and one random Slashtard named iabervon nailed it:

There's no one programmer who does the work of ten other programmers. One uber-programmer does just as much work as one ordinary programmer. It's just that the results solve ten times as many problems...

The best approximation is that each problem has a certain complexity and a certain size. The size determines how long it will take, and it doesn't matter how good the developers are. The complexity determines how good a developer is needed to make progress at all.

If you've got ten people essentially doing data entry, an uber-programmer may be able to eliminate the need for them to do that at all. If you've got ten developers working on some problem, an uber-programmer may be able to double their productivity. In either of these cases, the uber-programmer directly produces something that isn't part of the actual project, but the benefit to the project is on the order of ten average programmers' work. And, if the uber-programmer reduces the complexity of the problem to put it in reach of the rest of the team, no amount of ordinary programmers' work would benefit the project as much as the uber-programmer's contribution. Of course, if you require an uber-programmer to literally do the work of average programmers, there's no benefit at all.

No matter what kind of project you're working on, there's always a bit of tedium in programming... the uber-programmer is one who sees elegant solutions to complex problems, sees the connections to previous work, and thus uses modifications of older code to solve the current problem. They also create flexible code in the first place, so future similar problems can be easily resolved. They also know tricks on how to automate the repetitive aspects of the job to make themselves -- or others -- vastly more productive.

Unfortunately, in many situations its easier for the uber developer to do all the work, rather then wrestle with communication layers or management. If the uber developer is working with a small team, its probably more efficient for them to just do all the damn work... Its not an exaggeration to say its faster to write the code than explain to somebody else how to write it. In which case the uber-developer will certainly believe they do the work of ten people, and everybody else is an idiot... thus fostering Myth #2.

The solution is to recognize who the uber programmers are, and make sure the benefits of their labor cascades down to the rest of the team with as little effort as possible. Harder than it sounds...

On the other hand, if your project is not very interesting, not challenging, and doesn't lend itself well to automation, then whoever can type the fastest for the longest will be the most productive...

The solution here is to give your uber programmers an outsourcing budget. Let them hire a cheaper developer to solve the boring stuff, so they can move on to the more interesting problems that add value to your bottom-line.

Flickr FINALLY Adding Video

I'm all for this...

Techcrunch reported recently about a reliable rumor that Flickr will soon be adding video support, as well as their current photo support.

Some are upset about this change... noting that Flickr does photos well, so they should leave video to somebody who does video well. I disagree: I have a pro-Flickr account, and I like how their guest passes work...

I'm not interested in putting video of family vacations up on YouTube or Jumpcut for any goober to see... nor am I interested in forcing my friends and family into getting an account just so they can see my videos.

As far as I'm concerned, this feature is about 2 years late...

Identity Management Will Destroy Social Software

There's a meme growing on the web about social networks and identity... and I thought I'd inject my opinion before it grows into some kind of wretched mutant...

Firstly there was the long-standing opinion of Read/Write Web about the attention economy: concerning that the pages a person visits on site A is incredibly valuable to site B... Similar to how info about previous Amazon purchases would really help NetFlix, and vice versa. I liked their analysis, but I think they missed the point. Then there was the article in Wired about opening up Facebook: shouldn't Facebook be required to open up their APIs so people can copy them more easily? Um... yeah, that'll happen...

That was followed by the decent analysis by Nishant Kaushik from Oracle about how social software sites are popular only because there is no internet-wide identity management scheme. I don't believe Nishant went far enough... a distributed, user-centric identity scheme would destroy almost every "walled garden" social software application on the web... Which is exactly the reason why very few people would choose to implement it.

The true next generation of social software will embrace and share identity -- like del.icio.us does -- and thus enable users to replace old-school identity silos in a flash.

Think about it: if we had an internet-wide user-centric identity scheme (like SXIP/OpenID), then I could identify -- in one single place -- what my likes, dislikes, bookmarks, and contacts are. I wouldn't have to copy all my LinkedIn contacts into Facebook. I wouldn't have to tell every news site that I dig technology. I wouldn't need a hundred passwords... They would all just work together, and know me!

Or, more specifically, I would log in with my internet-wide username and password... then the apps would ask me if they could sync with my identity management server for contacts and interests. Then I would explicitly choose yes or no depending on what site I'm connecting to.

Personally, if del.icio.us offered some kind of SXIP/OpenID implementation, I'd use it in a flash... I already have all my bookmarks there, along with my tags. I've always said, my tag cloud says more about me than any customer survey I've ever taken... Del.icio.us is truly sitting on a goldmine that could destroy StumbleUpon, if they only knew how.

The Wired guys noted that it takes very little effort to duplicate 90% of what Facebook does... Nishant correctly states that the remaining 10% is totally solved with internet-wide identity management. I'm not sure if he's backing the right horse -- IGF style governance documents??? WTF??? -- but he has the right idea.

Frankly, Mark Zuckerberg, the wannabe-Steve-Jobs-CEO-of-Facebook, is really close to screwing the pooch on this one. He's sitting on a house of cards, not a goldmine. If Yahoo offers $1 billion for your company, frigging take it! Hell, if they offer $50 million its a bargain... unless you're certain you can con somebody else into offering more. Then jerk around the jerks for as much as you can get.

But don't take too long: once Ning dumps their NingID in favor of OpenID, Facebook's days are numbered. And frankly, if Ning was motivated, that could happen in under a month.

Sorry to bring this to your attention, capitan fleece, but somebody had to.

Further reading:

13 Rules For High-Speed HTML

The author of High Performance Web Sites just put together a list of 13 simples rules for high-speed HTML on Yahoo. Briefly, the tips are:

  1. Make Fewer HTTP Requests
  2. Use a Content Delivery Network
  3. Add an Expires Header
  4. Gzip Components
  5. Put CSS at the Top
  6. Move Scripts to the Bottom
  7. Avoid CSS Expressions
  8. Make JavaScript and CSS External
  9. Reduce DNS Lookups
  10. Minify JavaScript
  11. Avoid Redirects
  12. Remove Duplicate Scripts
  13. Configure ETags

Some of these are obvious... like using GZIP, reducing DNS and HTTP requests, and leveraging the Expires HTTP header. Some are really innovative, such as CSS Sprites -- which sound really cool but difficult to maintain.

Others seem like bad ideas to be, like using the data: URL scheme to embed base64 encoded images directly in the HTML. I'd only do that as a last resort to squeeze out every last drop of performance... its much cheaper and easier to buy new hardware or become an Akamai client...

In all, its a good checklist to run through on every page you have, in order to make it load as quickly as possible.

Recent comments