Twitter Performance Problems, The Root Cause

The performance and scalability problems of Twitter have been covered to death, so I won’t wax lyrical about the different reasons that the micro-blogging service has had performance and uptime problems over the last year.

With the advent of cloud computing and inter-connected web services, the requirement to have a good quality API has just about become a must have. One of the things that an API allows is new and creative mechanisms for users to consume and repurpose your service – which by and large is fantastic. Every now and then though, people will find a way to exploit a service to their advantage – usually financially driven.

In the case of Twitter, clever folk are using the service to ‘watch’ what discussions are happening on and around the internet about a given topic. Case in point this afternoon, I mentioned the phrase “WordPress” in a tweet and I suddenly received 10 new emails notifying me that random people I don’t know are now following me.

The fact that random people are following me isn’t the concern, it is that they automated that based on what I was disucssing in a Twitter conversation. The knock on effect is that those users will no doubt be following  hundreds or thousands of other Twitter users.

From an architectural point of view, this problem quickly spirals out of control as now every message that I write, generates a notification to be sent to those users. If they had a legitimate interest in following me, no problem at all but more than likely it will go completely unnoticed and the only thing that it has really achieved is increasing the load on the Twitter infrastructure.

If users continue to abuse this type of functionality, inevitably the Twitter folk will further tighten the screws on how many people you can follow per account. Of course, then the users abusing the service will start creating multiple accounts so they can get what they want – always looking for a way to side step the restrictions.