Crowd Computing and The Synaptic Web

A couple of days ago David Gelernter – a known Computer Science Visionary who famously survived an attack by the Unabomber – wrote a piece on Wired called ‘The End of the Web, Search, and Computer as We Know It’. In it, he summarized one of his predictions around the web moving from a static document oriented web to a network of streams. Nova Spivack, my Co-founder and CEO at Bottlenose, also wrote about this in more depth in his blog series about The Stream.

I’ve been interested in the work of David Gelernter for quite some time and thought this might be a good time to revisit some of his previous predictions. In 1999 he wrote a piece on Edge called ‘The Second Coming – A Manifesto’. While there are many pie in the sky things in there, I found some key takeaways that are highly relevant today:

Crowd Computing

18. But the Net will change radically before it dies. When you deal with a remote web site, you largely bypass the power of your desktop in favor of the far-off power of a web server. Using your powerful desktop computer as a mere channel to reach web sites, reaching through and beyond it instead of using it, is like renting a Hyundai and keeing your Porsche in the garage. Like executing programs out of disk storage instead of main memory and cache. The Web makes the desktop impotent.

19. The power of desktop machines is a magnet that will reverse today’s “everything onto the Web!” trend. Desktop power will inevitably drag information out of remote servers onto desktops.

20. If a million people use a Web site simultaneously, doesn’t that mean that we must have a heavy-duty remote server to keep them all happy? No; we could move the site onto a million desktops and use the internet for coordination. The “site” is like a military unit in the field, the general moving with his troops (or like a hockey team in constant swarming motion). (We used essentially this technique to build the first tuple space implementations. They seemed to depend on a shared server, but the server was an illusion; there was no server, just a swarm of clients.) Could Amazon.com be an itinerant horde instead of a fixed Central Command Post? Yes.

In order to make software and apps more intelligent, there is a vast amount of computation that needs to be done. Moving things into the Cloud will only get you linear improvements to computational scale. Meanwhile, there is a huge untapped potential sitting on the devices of the people interacting with the system. Why do Cloud Computing when you can do Crowd Computing?

At my company we send raw social media messages down to the browser, the browser then does natural language processing, semantic analysis and finally our StreamSense algorithms to discover trends in the stream. Once that’s done, the web browser will submit back results to our servers and the analysis is stored into our central analytics index. This is all happening seamlessly to the user of course.

Screen Shot 2013-02-05 at 9.13.08 PM Bottlenose Crowd Computing Layer

The move to Crowd Computing is a very natural progression of the web due to the lack of bandwidth in proportion to the available computational and storage power. This global network of Apps and Cloud servers has formed the backbone of today’s global digital infrastructure. The natural next step towards Crowd Computing is an improved ability for Apps to utilize their underlying hardware and an increased connectivity with each other (By means of P2P or through mediator nodes in the Cloud).

New Metaphors

28. Metaphors have a profound effect on computing: the file-cabinet metaphor traps us in a “passive” instead of “active” view of information management that is fundamentally wrong for computers.

30. If you have three pet dogs, give them names. If you have 10,000 head of cattle, don’t bother. Nowadays the idea of giving a name to every file on your computer is ridiculous.

Screen Shot 2013-02-05 at 9.16.30 PM Not only is this true for the ‘file’ metaphor, it is even more true for the ‘page’ metaphor. Ultimately all these metaphors are based on physical objects, the problem is that in the digital world these objects behave in the opposite way. Physical objects decay when you touch them, but digital objects multiply when touched.

Some of today’s metaphors however do a much better job at reflecting the abundant and infinite nature of the web: Streams, Filters, Attention, Channels, Contexts, Connections, etc.

The Synaptic Web

41. You manage a lifestream using two basic controls, put and focus, which correspond roughly to acquiring a new memory and remembering an old one.

43. A substream (for example the “Fifth Avenue” substream) is like a conventional directory — except that it builds itself, automatically; it traps new documents as they arrive; one document can be in many substreams; and a substream has the same structure as the main stream — a past, present and future; steady flow.

The act of inserting a small piece of content into the stream – for example a tweet or a like – can be compared to a neuron being fired in the brain. The signal is then broadcasted to about 1000 other neurons through synapses (connections). These neurons may then choose to fire to their connected neurons and so on. This is similar to the way messages travel through social media networks. And of course after your friend has posted LOLCAT number 300 you might decide to re-arrange some of your synaptic connections.

Querying the stream is like recalling a memory, although our ability to query the stream is quite primitive. Querying ‘videos of cats playing the piano’ is probably far from higher-level reasoning.

dslc0020Neurons in the Human Brain (image credit: Riken Institute)

The human brain holds about 100 billion neurons, of which each has about 1000 synapses. Each neuron can fire between 0 and 200 times a second which makes the total potential throughput of the brain mind boggling. In non-parallel computer terms the ‘clock speed’ of this CPU would be 20 million GHZ which is about a million times faster than the CPU in my laptop. This total capacity is however never utilized all it once (you would in fact die), instead there are periods of high intensity bursts (in which neurons seem to behave according to mob mentality) and periods of low activity, for example when sleeping.

The web is no longer about documents and pages. The Semantic Web as envisioned by many in the last two decades is inherently flawed. The web is not about knowledge, facts or data. The web is about people, activity and connections. This emerging Synaptic Web is the beginning of a planetary intelligence. The Synaptic Web is our collective stream of consciousness, although the global brain itself is far from conscious yet. It cannot reason or think yet. The global brain is still dorment, like a baby in a womb, slowly developing its neocortex and silently observing us…

Until it wakes up…

Posted in Uncategorized | Leave a comment

Beyond the Knowledge Graph

Google has finally digested the acquisition of Metaweb and has radically enhanced its search results by utilizing what Google calls The Knowledge Graph. When searching on Google now, search terms are refined into more specific versions of the meaning: Did you mean Java the island? Or Java the programming language? To me this is one of the most powerful things of the Google Knowledge Graph.

What it also does is pull out facts about the topics you’re searching for. An example used in their promo video is reading facts about Leonardo DaVinci. Or another example used is “How many Nobel Prize winners were female?”. In my mind, these examples strike at the heart of something that has always been bothering me about the Semantic Web.

4499404_e2fb14178a_b

The Semantic Web

The Semantic Web is the vision that all knowledge is stored into a structured form and consumable by computers. This idea has been around for a long time, but most of the execution so far has been limited. Also, the open standards like RDF and OWL have never really taken off beyond the academic world. I personally don’t think they ever will, because they are incredibly unpractical and centered around a “Web of Documents”.

Even though I greatly respect the technology behind the Knowledge Graph (I have been a fan of Freebase for years, and have deep respect for their platform) – I still think there is a fundamental part of the equation missing. It’s something that Google often fails to get right: Human Interaction.

The Stream

There is a sea change happening in the Web and how we use it. Itʼs an evolution to a second phase of the Web – the real-time Web, or what I call “the Stream.” In the Stream, the focus is on messages not Web pages. These vast amounts of messages are generated by social interaction, by conversation, by attention, by ideas, by little chunks of thought unleashed into a gigantic stream of data.

Thanks to the Stream the amount of knowledge is exploding. There is a long tail of knowledge around every imaginable topic and our attention is spreading across it every thinly. In the long tail of knowledge one man’s junk is another man’s treasure. Knowledge that has previously been at the head of the long tail is now becoming obsolete knowledge – or “Obsoledge” – ever faster.

Who really wants to know what percentage of Nobel Prize winners were female? Or when do I ever want to figure out the birth date of some dead dude? These geeky factoids are becoming increasingly irrelevant. So where does the knowledge in Google’s Knowledge Graph come from? Does it come from geeks that are eager entering facts? Or does it come from corporate curated databases?

We need to be able to harness the Stream in order to query the long tail of knowledge. What is going on right now around Leonardo DaVinci? Is anyone recreating his cool flying devices? Can I buy them? Who won a Nobel Prize but shouldn’t have? Any pictures of the after party? Are there any new “prizes” that compete with the Nobel Prize?

To succeed in making the Knowledge Graph or Semantic Web usable for ordinary mortals, we have to drop the notion of a perfectly modeled graph of knowledge. We need to start looking at the Social Graph and start listening to the gigantic Stream of human activity. This means finding new ways of utilizing social activity data to generate meaning, liberating data from their data silos, providing more structure and open standards around social data interchange, etc.

We need a web in which the Stream is more meaningful, a web in which information finds you, a web in which data flows based on attention and intent, a web in which our thoughts are interconnected like neurons in a brain.

We don’t need a Semantic Web.

We need a Synaptic Web.

Posted in Uncategorized | Leave a comment

The Rise of JS and the New Server-side

I started writing this article in my excitement of Server-side Javascript (SSJS) and how I’ve become increasingly interested in Javascript (JS) over the past years. But at the end of the day, Javascript is just another programming language, its importance is defined by the frameworks and applications that are built on top of it. But while I was writing this, I noticed that together with the rise of Javascript, something much bigger is happening in the way services are and need to be structured.

Javascript in the Browser

A little more than a year ago, I wrote about the grim future of Adobe Flash. I think by now, there are even more signs that Flash is dying in it’s current form. Flash got a big stab in the back recently by Youtube when they started supporting the HTML5 video embed and also Apple is once again not supporting Flash on the Apple iPad.

Javascript is getting faster and stronger in the browser. A good example is the powerful, open-source V8 Engine developed for Google Chrome. Also, mobile devices are increasingly centered around Javascript, all Palm Pre apps are written in Javascript; the iPhone Google App is pretty much a JS interface in Safari; And there are cool new frameworks out there that allow you to write native iPhone/Android/Desktop apps completely in JS.

A simple client-server web application using the LAMP stack.
An oldschool client-server scenario with a web application running on the LAMP stack.

Javascript Beyond the Browser

At the end of the day JS is just a programming language, it’s success is determined by the amount of adoption and the problems it can solve. So far most of the problems solved by JS have been in the browser. But this is about to change.

Server-side Javascript is nothing new, but it’s not common yet. There are many open-source frameworks sprouting up now and I think the rise of cloud-services like the Joyent Smart Platform will definitely help SSJS taking off. Also, there are some serious economic benefits to having JS be the only programming language used in a web application. Do you know any programmers that don’t know how to write HTML? How about Javascript? JS and it’s relatives – ActionScript and Java – are very widely known by developers. This means that finding and training JS developers is easier.

These SSJS frameworks need to accommodate very different use-cases than their client-side counterparts (jQuery, Prototype, …). A SSJS framework shouldn’t care about the DOM, but it should care about things like IO / file access, network and database connectivity, template rendering, communication with existing dynamic libraries, etc. The server-side use-cases are completely different!

The Migration to Client-side

For the past years, I have been involved in scaling a huge service called Smart.fm. Over the course of 2009 we’ve made some big adjustments in our technology strategy. We’ve been offloading more and more controller logic into the browser using jQuery/Javascript. This also means that we’ve had to beef up our API and make sure all data can be accessed smoothly using JSON feeds. This has given us huge benefits in performance, modularity, manageability and user experience. But when looking at top-sites out there like Facebook and Google, this seems to be common practice nowadays. More and more logic is now running in the browser.

A typical Rails stack application.
A typical client-server scenario with a web application using the Rails stack.

So all of this has got me thinking: what does the server-side still mean for a web application?

The controlling of the UI flow and interactions is now done by Javascript in the client. This means that access control is purely done by the API, the client will not get the data it doesn’t have access too. This means that all access control and data routing will be controlled by the API.

What about server-side rendering of templates? Surely we need this for the search engine bots, but do we really need to treat them the same as users? Search engine bots like the Google Bot need to be able to scrape server-side rendered HTML information and will not execute any client-side logic. But as simple keyword search is becoming less relevant, we need to start building different interfaces for machines. Why not render some simple HTML for those dumb Google Bots and not worry about it in our UI? Oh and while we do that, render something more semantic for those smarter bots!

Architecturally, we can define these components in a modern web application:

  • An API that serves as an interface to the business models and provides access control
  • A user application that runs completely in the browser that uses the API for access to the domain’s data, but also interfaces with a multitude of other JSON API’s (Google Analytics can also be seen as a one-way JSONP call).
  • Simple server-side rendering capabilities for delivering the user application code and for serving semi-structured content for the search bots.

So when all rendering and UI flow logic moves to the client, what responsibilities remain for the server? It is very important to ask this question when you build a new server-side framework. What problems does it need to solve? Most server-side web frameworks are built around the increasingly irrelevant Model-View-Controller (MVC) pattern.

A Next-generation Architecture

Now that the responsibilities of both the client and the server have changed, we need to build new frameworks accordingly.

An example next-gen architecture

The browser, technically known as a ‘user agent’, needs a framework that takes care of all the UI flow. This means that we need the MVC pattern running on the client. This ‘user agent framework’ needs controllers that can render views and it needs a comprehensive routing system based on anchors.

As a business, now that more of your code (a.k.a. intellectual property) is running openly in your browser, where does your strategic advantage come from? Well, it comes from making your backend very intelligent.

The server, who I like to call ‘machine agent’, needs a very solid Application Programmer Interface. The sole purpose of this API is interfacing with different user interfaces (web, iphone, ipad, …). However, it should have a separate facility to deal with other ‘machine agents’ like Google Bots. Other servers that interface with your system have very different needs than third-party user interface applications. An example of this would be API calls that perform bulk operations (like Twitter’s mass-following call).

In order to deliver smart services that can compete, machine agents need to be able to analyze the state of their internal universe (business models) and act on them. This could be a simple iPhone push notification that gets sent after a certain event happens, or it could be after the completion of a statistical analysis. Web backends need to become more event-driven (like Javascript) because a simple cronjob will no longer suffice.

Machine-agents need to be able to integrate with other services. Sometimes this means that they need to be able to fetch many feeds from a single service. I consider this to be one of the hardest technical challenges of most projects I work on. For example, if I want to do something semi-intelligent with someone’s Twitter activity, I will need to first fetch all his data, import it and then analyze it. This is very expensive and I haven’t found any economically viable solutions yet. Of course no one wants to passively fetch feeds and then analyze them, it would be much better to do something like server-to-server long polling (e.g. listening for Tweets with a certain keyword), but sometimes you have no choice: you have to aggregate.

Conclusion

The use of Javascript on websites has been unstoppable and now it has spread to other platforms like the Palm Pre and iPhone. Javascript is a programming language that’s here to stay.

It is now common practice for web application developers to off-load a lot of work to the browser. This migration to the browser together with the changing technology landscape requires us to re-think web application architectures and frameworks.

Posted in Uncategorized | Leave a comment

Rethinking the Knowledge Economy

During the few ‘off hours’ I have, I love thinking about innovation and technological change in the grand scheme of things. In fact, a couple of weeks ago I did a weekend trip to New York on my own budget to attend the Singularity Summit 2009 – a conference of futurists. In my recent thinking I’ve stumbled upon a nagging little problem: The concept of the Knowledge Economy is very ambiguous and ill defined.

When I look up the definition of Knowledge Economy on Wikipedia, there is already a big alarm bell going off in my head when I read: “In a knowledge economy, knowledge is a product, in knowledge-based economy, knowledge is a tool.”. The words product and tool treat knowledge on par with physical goods, which in my mind is the reflection of a fundamental disconnect.

Knowledge

In the academic world there is no real consensus on the difference between information and knowledge, a definition that’s not at all trivial. Asking the question will most likely yield different answers depending on which ‘field of expertise’ you consult (management, philosophy, computer science), but there is a common way of explaining the concepts of data, information and knowledge on a continuum:

  • Data is raw facts
  • Information is data that is organized, analyzed or placed into a certain context
  • Knowledge is internalized information: skills, experience, cognition, etc.

“Knowledge is the whole body of cognition and skills which individuals use
to solve problems. It includes both theories and practical, everyday rules
and instructions for action. Knowledge is based on data and information, but

unlike these, it is always bound to persons. It is constructed
by individuals, and represents their beliefs about causal relationships”

(Probst, Raub & Romhardt, 2000)

There is also a very popular view (by Nonaka & Takeuchi) in which knowledge is divided up into explicit knowledge and tacit knowledge. But this widely accepted view has now come into dispute because many claim the concept of explicit knowledge is no different than information.

No matter what stance you take, it’s clear that knowledge always has a certain context associated with it. On one end of the spectrum this could simply mean that certain chunks of knowledge are useless without other chunks of knowledge (still explicit). In other cases it might be that the knowledge is completely useless without a lifetime of experience or skills (tacit).

However, I do think it’s important to realize that while tacit knowledge is a lot less ‘portable’ than explicit knowledge, this could change very quickly with the advent of technology. With the existence of zero-cost communication, learning-enhancement software and artificial intelligence, tacit knowledge is also becoming more ‘portable’.

Many thanks go out to Jozua Loots who helped me with the question “What’s the difference between knowledge and information?”. All done through an awesome new (synaptic) Q&A service called Aardvark.

Informational v.s. Physical

To understand the fundamentals of the problem, we have to take a look at the difference between information and physical objects. Physical objects abide by different laws than information. A physical object can only exist in one place in one time and it deteriorates when used or touched. Information on the other hand, can exist in many places at any time and multiplies when touched.

Picture 26

Thanks to zero-cost communication, the replicating nature of information has showed itself over the last decade. There are now vast amounts of knowledge (and obsoledge) being generated every day, making many derivatives of information (content, knowledge) a commodity. Kevin Kelley explains this well in his essay ‘Better than Free’, where he compares the internet to a giant copy machine where the copies drop in value. Interestingly, when those copies become abundant, the value starts shifting to what’s scarce: the attention of people. This is where the concept of the Attention Economy starts.

A Non-Economy?

If information is so fundamentally different than material goods, you can start asking the question: Does the term ‘economy’ apply to knowledge at all? Let’s take a look at the wikipedia definition of the word ‘economy’:

“An economy is the ways in which people use their environment to meet their material needs. It is the realized economic system of a country or other area. It includes the production, exchange, distribution, and consumption of goods and services of that area.”

And here is the definition of an ‘economic system’:

“An economic system is the system of production, distribution and consumption of goods and services of an economy. Alternatively, it is the set of principles and techniques by which problems of economics are addressed, such as the economic problem of scarcity through allocation of finite productive resources.”

Both definitions are inherently bound to ‘goods’ and the fundamentals of finite production. The problem is that with information, there is infinite free replication. This explains why it is so hard to use traditional methods of economics to measure and understand value created by information. But fortunately there is hope in the definition of ‘economics’ itself (by Lionel Robbins in 1932):

“The science which studies human behaviour as a relationship between ends and scarce means which have alternative uses.”

Because even though the characteristics of information/knowledge are so different , the economic fundamentals of abundance and scarcity still apply.

Informational Drivers of Economic Growth

The influence of information on wealth creation is quite complicated. The biggest mistake people make is treating information (or it’s derivatives) as a physical good that can be traded. So if you can’t sell it, it has no value? Yet information plays a profound role in driving economic growth.

Hans Rosling, a Swedish econometrist has given several TED talks in which he showed how developing countries have been caching up with great speeds. Every booming developing country had it’s own drivers of growth, but one can imagine that a common driver would be the availability of ‘know how’. This ‘know how’ took the Europeans centuries to develop and apply, but for developing countries this was readily available and could be applied fairly quickly. This application of explicit knowledge set off the main driver of growth: change. And when you apply new knowledge to a country that needs to build things from scratch, you get a rapid rate of non-incremental change.

On the scale of an entire economy, it’s really the non-incremental change that matters. An example of normal incremental change could be aesthetics, the improvement of physical products.  Another example would be a well-oiled service industry that services existing markets. All of these dwindle in comparison to the amount of value created by fundamental change. Real innovation will create and destroy new markets (e.g. telephony, online advertising, social networking, etc) whereas incremental change merely optimizes existing market dynamics.

The rapid shift from physical systems to more informational systems – informationization – goes hand in hand with non-incremental change. When a system becomes more informational and has less physical obstacles, changes can happen more quickly. And these changes are non-incremental, meaning that informational systems will have more paradigm shifts, are less predictable and have more volatility. Nassim Nicholas Taleb (NNT) explains this in a different way in his famous book The Black Swan, whereby he defines an Extremistan (informational-law world) and a Mediocrestan (physical-law world).

To summarize:

  • It’s not the explicit knowledge that creates value, but it’s rather the flow and application of it.
  • Non-incremental change is the main driver of economic growth
  • The more informational a system becomes, the more non-incremental changes will happen

Note: A nice example of informationization can be found in the biography of Nikola Tesla, one of history’s greatest inventors. Tesla had a special brain condition where he could visualize and iterate his inventions in his mind using his photographic memory. This allowed him to innovate at a very rapid pace, because he did far less physical experiments in the innovation process.

Stimulating Non-Incremental Change

Some governments, like my own (the Netherlands), come up with special ten year action plans that try to create a vibrant ‘Knowledge Economy’. What should these action plans entail and how relevant is the knowledge aspect of things?

The first thing that needs to happen is the stimulation of informationization. This requires removing physical constraints in for example bureaucracy. Many governmental organizations still require you to handle paperwork with real paper or require you to unnecessarily interface with a person. Also, there needs to be a general reduction of the amount of bureaucracy. This could be done for example by reducing the amount of certifications required to start a certain (informational) business.

Education needs to stimulate independent thinking. They need to stimulate their students to do new things (sponsor adventurous travels?), but more importantly: they need to shift focus from ‘knowing’ to applying knowledge and using creativity.

Corporations need to be formed ever quickly, but more importantly, they need to be dissolved quickly too. Companies – that are getting smaller and leaner – must be able to fail early and often in order for real innovation to happen. This also means that you need a culture that can deal well with failure and makes sure that the people involved don’t have to deal with ‘face loss’. On the flip side people need to be rewarded when they are successful, this might mean having a more loose taxation system for the wealthy. One idea here could be to allow tax-free re-investments of earned capital to stimulate successful entrepreneurs to become angel investors.

Which brings me to entrepreneurship: you need a very vibrant investment climate that has VC’s and angel investors that invest in bold ideas. Many countries outside of the US cope with a sickening amount of risk averseness. When you present a prototype to a European investor, they don’t ask you when you will generate revenue, no, they will ask you: “when will you break even?”.

Summary

The economics of a world that is becoming ever more complex are not as simple as they have been. The concept of a “Knowledge Economy” is outdated and is not factoring the new dynamics of informationization. The real driver behind economic growth is non-incremental change, something that is catalyzed by informationization. In order to gain economically, governing bodies, companies and people need to stop resisting informationization and go with the flow.

Posted in Uncategorized | Leave a comment

The Synaptic Web

The Real-time Web (RTW) is buzzing all over the place, but I’m amazed by the amount of different interpretations of it. The RTW – or better said: large scale activity streams – are indeed reshaping the entire business and technology landscape, but how? This will be the first post of a series of articles that will be centered around this topic.

Micro-blogging (status updates) are the most visible form of the RTW, but in parallel we already had things like: RSS feeds, social bookmarking and video favorites, etc. Combine them all together and you have your first lifestreaming services. It’s important to note that lifestreams are the first form of activity streams that have more meta-data than for example a Twitter stream. A lifestream can for example include tags about a certain video that was favorited.

Instead of using the buzz word RTW, I’d prefer using the term Activity Stream. Why? Because the real-time aspect is only one half of the story. The important part is the ‘activity’.

Because of several intersecting trends, we have now arrived at yet another paradigm shift that will radically change business and technology.

The Attention Economy

Twitter is very cool and all, but can it be monetized? Hell yeah!! It’s like in the 19th century when oil was discovered: what the hell do we do with all this black stinky stuff? Well, it burns, so what? Activity is the new oil and Twitter will be the new ExxonMobil. Now it’s time to invent the appliances that run off of it.

In books about Google, like John Battelle’s The Search, the big take away is that Google is uniquely positioned between supply and demand. Well, they are, but they’re still very much in the middle. Like in most of their products, Google misses out on the ‘social stuff’. Having a direct activity communication line with demand is what Google is missing. In the Twittersphere, the dynamics are such that the supply trickles up to the demand (the people) in a very targeted way, much more targeted then Google Search.

This is where the true Attention Economy takes shape. When people explicitly state things about their activities, they leave an attention trail. This trail can be used to actively push relevant services to people.

But there is a caveat, in order to start using this ‘attention trail’, we need two things:

  • We need a lot more activity. Right now we only have activity going out that is highly explicit, like the favoring of a picture. But an attention trail should include much more activity events, like “How many seconds did you spend looking at this picture?”. Right now, most systems have separate mechanisms for measuring this implicit attention data (i.e. analytics), but one can imagine that this data will be highly desired to do internal content recommendation.
  • In order to properly analyze the attention trail, machines need to be able to understand each individual event. For example: “What was on the picture?”. One way of accomplishing this is by having proper meta-data on activity events. Something I have been struggling with in my Kakuteru project and the very thing that prompted me to build my own aggregation framework.

The Semantic Web

Data mining – the extraction of structured data from text – has been around for a long time. Over the past years several web companies have started doing this by means of open APIs, most notably Zemanta and OpenCalais. But interestingly, someone at Zemanta told me that most of their API usage is for extracting facts from micro-content like Tweets (so I imagine that they will adjust their product accordingly). Unfortunately, many view extracting facts from micro-content as more challenging than large formal texts. In a way it is, because a Tweet for example has a lot of slang and abbreviations in it, but you can make things a lot simpler by mixing in nano-formats. You can for example imagine that a hash-tagged tweet like this is easy for a machine to parse:

“I’m boarding my #flight KL862 to Amsterdam”

So if you combine status updates with the extraction of structured information, you can already imagine an Agent or Twitter-bot providing some useful information in return. In this case for example, it can provide information about the weather in the destination city. Or it could direct-message you if people in your social graph are boarding the airplane as well. These Semi-Intelligent Agents can be considered the first step towards the reasoning stage of the web.

The Synaptic Web

This word came on my radar the first time I was looking at a stream integrated commenting plugin for WordPress called JS-Kit Echo. I think the word accurately describes what’s happening: the neuronification of the Web (thanks @mcmurrak). Futurist Kevin Kelley has written about this phenomenon as well. In essence, the connected nodes (people and computers) are an emergent global organism. But very concretely, the word “Synaptic Web” captures all important properties: ubiquity, meaning and interconnectedness (social).

So what is this Synaptic Web, and how will it change everything? Well, that remains to be seen and it’s something I plan to elaborate on in many articles to come. But here are some appetizers for thought:

  • Google’s Adwords were a very effective form of advertising because they were so targeted, but what if advertisements become so targeted that they are essentially offers/solutions provided to you? Will we still think of them as advertisements? How much are these ‘advertisers’ willing to pay for an advertisement that is 90% likely to trigger a full engagement with the consumer?
  • Demand for something specific is always temporary. When a big clustering of needs is ‘detected’ small nano-companies can be formed to fulfill these needs until they are gone.
  • Disambiguation is something that has been key to innovations in semantic technologies and will be very important for the Synaptic Web. Are we talking about the same thing, place, person? Machines need to be able to understand these things and serious innovation is needed here.
  • When machines are able to translate a concept like ‘NYC’ to a specific location with coordinates, they might become an important tool in quickly transferring concepts to each other. So instead of communicating in text, audio or video we could communicate in actual concepts. Of course this all depends on how we interface with the web.
Posted in Uncategorized | Leave a comment

A Twitter Bot in One Line of Code

Yep! It really IS possible with only one line of BASH (Terminal) code using the new Twitter Streaming API. Using a series of chained commands, this Twitter Bot listens for occurrences of ‘one line’ in the Twittersphere and will follow that person.

Here’s a version with some extra space for readability (does not really work with this extra space):

Note: This is just to display how more easy and easy it is to write bots/agents. Yes, this code consists of multiple commands and is over 400 columns. No, I don’t believe in measuring any software in LOC.

Note2: Many years ago I also wrote an IRC client in BASH

Posted in Uncategorized | Leave a comment

5 Short Sci-fi Art Videos

Some nice arty technology clips that I’ve collected from my attention trail.

Artificial Paradise Inc

By Jean-Paul Frenay (via Humanity+). A good Nano-tech teaser.

Uploaded Companies

By Jean-Paul Frenay

As One

By Makoto Yabuki

What’s in the Box?

This short clip got a lot of attention, because it’s made by a young Dutch physics student named Tim Smit, with a budget of 125 dollars and a pizza.

[youtube=http://www.youtube.com/watch?v=IU_reTt7Hj4&w=560&h=340]

La Jetee

One of the oldest and most famous science-fiction movies of all time. Black and white photos that picture a genius story. The movies 12 monkeys and many other are based on this.

[youtube=http://www.youtube.com/watch?v=3RvmJan17q8&w=425&h=344]

Posted in Uncategorized | Leave a comment

Cutting Your Losses Early

My little pet-project Kakuteru.com (lifestreaming service) is hereby declared dead. Over the last months, I have put effort in rewriting the core architecture of Kakuteru, the activity stream aggregation framework, but I’m aborting it now for several reasons.

To my dear alpha users: I sincerely apologize and thank you for your support over the last quarters. I will continue to run the service for another month and you can download dumps of your data. Also, I will make available all source-code and try to wrap some intelligent components into Ruby Gems.

The Real-time Web is still in it’s Infancy

With this comes great opportunity, but also great challenges. When I started Kakuteru, I made a quick mashup using Friendfeed’s API and I have complained several times about the lack of meta-data available in it’s stream. After looking around I decided to start doing my own aggregation, pubhubsub was not yet mature (and still isn’t). Gnip was a very interesting service, but it didn’t support enough different services.

Aggregating all those feeds from passive API’s is very expensive, I wrote a framework in Ruby that is pretty damn solid using Nanite (fabric of ruby agents), but you still have to deal with rate-limiting, format changes, scaling, etc.

In other words, getting a huge pipeline of activity stream updates is NOT a commodity! Not yet anyway. When looking at prices charged by services like Gnip and SuperFeedr, you can see this is obvious. Specifically looking at Gnip, it seems they’ve gone back and forth a bit between different business models.

I am very confident however, that companies like Gnip will be major players in the Real-time Web game.

Synaptify

One of my big personal goals is to find ways to create value by marrying the Semantic and the Real-time Web. In my daily work, as the CTO for Smart.fm I’m doing a lot of innovation on these fronts with a great team of superstar developers. However, we are all busting our asses, so there is not a lot of time for big undertakings like Kakuteru on the side.

However, I do intend to keep on hacking and build mashups in the wee hours. Specifically, I want to create Nano-startups that make use of this emerging ‘Synaptic Web’. So for starters, I’ve rebranded my blog to Synaptify.com, where I will push out well-digested startup ideas, semantic/real-time hacks and general technology thinking.

Posted in Uncategorized | Leave a comment

The Failed Understanding of the Information Age

Note: this is a crazy rant I had lying around on my Notepad that I worked on in transit

It’s good to be back a while in my home country, the European-average Netherlands. Since last December I’m renting a pimping sublet in the center of Amsterdam. The owners are trying to pursue their dream in Hollywood (good luck!) and I’m guarding their purple walled pimpmobile. Unfortunately, they cancelled their DSL subscription last month which I found out only recently. As soon as I could I called the DSL company and asked to ‘take over their subscription’, but unfortunately that was a naive dream of mine. Procedure dictates that the DSL modem has to be sent back by the old customer, I would be treated as a new customer and receive a new DSL modem (probably the same one). No mechanic has to show up. Average time of delivery: five weeks.. FIVE WEEKS Yes Sir, there is a mandatory one week ‘are you sure’ period mandated by the government and four weeks is “the normal waiting period”.

So now, as an internet refugee, I’m writing this in the library of the Utrecht University campus. Working here is awesome, there are huge tables and spaces. Beautiful modern architecture that inspires creativity. All funded by the government as part of their plan to make the Netherlands ‘Kennisland’ (Knowledge-land). All of this makes me think… Does the Dutch Government really understand the changes happening in the world? To them, a knowledge economy seems to mean moving away from relying on industrial output and stimulating a service industry with highly educated workers. But aren’t they missing something?

The Age of Extremistan

One of the biggest changes happening in our world today is the shift away from the physical (industrial era) to the informational (information era). Yes, it is now possible to generate tremendous amounts of value all virtual. How is that possible? Simply think about a car factory. The car factory started with big groups of humans assembling every little component, later things got automated more and more and now programming robots is much more valuable than touching any material objects. Factories are becoming more modular too, with generic purposes machines that can shape objects based on 3D CAD-designs. Now imagine the most generic purpose assembling device, yes! It’s the Star Trek Food Dispenser! :-] Science fiction or not, molecular manufacturing will be the end of the true end of the Industrial Age. It’s all nanotechnology from there. And as these changes happen, more and more physical objects will become informational.

As more and more things get informational, new laws apply. Information behaves very different than physical objects, but is unfortunately often treated the same. The current shake-up of the music industry is a great example of how traditional models of scarcity fail. Kevin Kelley explains it well in his essay that asserts that everything that can be copied will drop in value.

Nassim Nicholas Taleb in his book The Black Swan makes a distinction between Mediocrestan and Extremistan, respectively meaning the physical and informational worlds of probability. Extremistan has the tendency to be much more volatile, quicker changing and unpredictable.

It’s the activity that matters

One thing that bothers me about the collapse of the financial industry is that people scream ‘back to basic’, and shift their focus back to the tangible. Complex financial instruments are not necessarily bad, massive speculation without transparency IS.

People fail to understand that every industry is essentially a bubble and has a limited lifespan. The music industry is a great example. There was a time, in the beginning of it’s forming where a pop-artist could produce one hit song and live a luxury life for the rest of their life. After a while, this was becoming an expectation throughout the entire music industry. But now that money can no longer be charged for the actual content, the sector is imploding. In other words, you need to constantly work your ass off to make a buck. Perform, sign t-shirts, engage constantly…

Every industry has a lifespan that is based on technology, in it’s early stage people can make money in a highly scalable way (one day fly), but there comes a point where constant change (creative input) is necessary. Every industry!

And guess what, while the pace of innovation is speeding up, industries and companies will have ever shortening life spans. (Check out my crazy illustration below :)

What can organizations do?

The real competitive advantage for any (corporate) entity is the ability to change. As things are speeding up and markets expand and contract more rapidly, it’s all about being agile. This means that larger organizations are at a disadvantage.

  • Move away from a monolithic structure to a fabric of smaller cells that work autonomously
  • Give those cells more responsibility and shift to an ‘entrepreneurial workforce’
  • Destroy the clustering of specialization, every cell should have multiple disciplines
  • Provide the cells with extreme communication tools
  • Abolish salaries, be more like an incubator
  • Embrace new technology like never before

What can Governments do?

(Image courtesy of the Long Now Foundation)

If Governments really want to succeed in the information era, they need to provide the tools for rapid change. Stimulate velocity at all costs. This will require radical change and cut-throat free market capitalism. Yeah, I said it!

  • Invest heavily in digitizing all public services, most importantly: Tax bureaucracy, Chamber of Commerce
  • Make sure you can register and dissolve corporate entities within seconds (through an API)
  • Invest heavily in both digital and physical infrastructure
  • Reward taking risk and entrepreneurship, at cost of social welfare
  • Give up taxes on many things. Already, the Dutch Government doesn’t tax ‘electronic services delivered to non-European countries’. Added informational value is hard to understand and will only get more complicated in the future.

Coming from a European country and having written some of my crazy ideas here I realize that it’s going to be a bumpy road. Governments, especially European ones, will move incredibly slowly.

In a way that’s sad, because the changes our world is going through will require some serious governance. Bio and nano- technology can have monstrous consequences when not controlled correctly. Things might become crazy – sooner than we think :]

Posted in Uncategorized | Leave a comment

Kakuteru.com, Shake that Cocktail!

Hey everyone, it’s time for a little Kakuteru update. It’s been a couple of months since I’ve started building Kakuteru. Since then a lot has happened!

Last month, I’ve relocated back to Amsterdam to pursue one of my long-term goals: ‘the startup’. Together with Peter Tegelaar, I’ve rented a nice apartment in the centre of Amsterdam. Here we will work on our stealth project, Reccoon. And for the time being, I will remain a Technical Advisor for iKnow! – the global learning platform (and remain highly involved).

Please note that Kakuteru is not my full-time engagement, however, I think there is a lot of potential for issekinichou – killing two birds with one stone. On some of my other projects, there are huge overlaps in terms of technical problem solving. I would like to do a write-up about issekinichou sometime soon, as I think it’s a very important strategic meme that helps deal with the ever increasing complexity of the world.

Okay, let’s get back to Kakuteru. First of all, I’m very satisfied with the enormous amount of excitement and buzz Kakuteru has generated in the web-tech crowd. Also, I’ve got an overwhelming amount of invite requests and I just send out a batch of 400 invites (through Twitter and E-mail). But of course, Kakuteru is young and there is still a lot to be done. Let me outline some of the upcoming/desired changes.

More Aggregation!

Right now, Kakuteru’s backend lies solely on FriendFeed. You cannot use Kakuteru unless you have a FriendFeed account. I want to change this. It’s not that I dislike FriendFeed, au contraire, I think their service kicks ass. For Kakuteru I’m missing something very important though: massive amounts of meta-data.

Luckily, I have to solve some of these aggregation problems for my personalization-focussed startup Reccoon. For Reccoon we’ve been toying with several ideas (as a business) and right now we’re investigating something that requires us to create a massive aggregation framework. I intend to plug Kakuteru into this framework so that more (semantic) data is available around activity objects and streamers.

Also, I’m looking very forward to the work of the DiSo project and intend to fully support and adopt standards like the Atom Activity Extensions. There is a lot of cool innovation possible on the activity stream front.

_Note: To my astonishment I’ve been blogging about some Activity-platform ideas before: Startup Opportunity: the Activity Platform

More Customization!

Right now the default Kakuteru theme lacks charisma. So far you’re able to edit your CSS and add additional Javascript, also you can add your own domain, but I want to stimulate more radical creativity around lifestreams. This week I’ve added the ability to control the big layout blocks in Kakuteru, so it should be easy to re-arrange things and add extra content. Soon, I will also add the ability to customize individual activity-updates (e.g. photo, video, tweet).

Now, to add some kick-ass default themes, I have another trick in mind ;] (more about that soon!)

An Open Venture

As you might know, Kakuteru’s source is fully available on Github. But I’d like to go beyond open source and make Kakuteru an Open Venture. I’m not sure how to accomplish this, but I’m willing to give ownership away if I think it makes strategic sense. So if you think you’re up to the challenge and help me make this a kick-ass side-venture, contact me!

Improvement suggestions? kakuteru.uservoice.com

Posted in Uncategorized | Leave a comment
  • About this Blog

    Hi, I'm Dominiek. I'm the CTO & Co-founder of Bottlenose.com. This blog is about Technology, Innovation and the Web.