Bartek Nowotarski

The Amazon interruption was the computing equivalent of an airplane crash. It is a major episode with widespread damage. But airline travel is still safer than traveling in a car — analogous to cloud computing being safer than data centers run by individual companies.

— Lew Moorman, chief strategy officer of Rackspace, http://www.nytimes.com/2011/04/23/technology/23cloud.html

All of our customer acquisition has been free — through social media, blogs, and the press. We don’t pay a dime for advertising. We also use business partnerships with Motley Fool and the credit-report companies. And Apple has given us a lot of free advertising as a featured application.

— Aaron Patzer, Founder of Mint.com
found in Inc. Magazine

Captcha Impossible

Code release process in Facebook

I’ve just come across interesting video in which Chuck Rossi, Core Release Engineer in Facebook with 27 years of experience as an engineer in Silicon Valley, talks about release process in Facebook and how they manage to push updates every day.

If you don’t have time to watch video here’s a list of the most interesting things I’ve noted watching it:

• employees at Facebook are using latest.facebook.com (test branch) so as they are using Facebook they actually testing code before it’s release,
• they use Facebook groups to communicate, one of them is named: “Bugs forum”,
• the whole world is using Facebook so they don’t use production for testing,
• they also use IRC in communication between engineers and IRC bots,
• they have a special tool to track PHP errors and exceptions (with a call stack) where they can see diagrams of frequency of error occurrence and where it occurs (mobile/api/…),
• they have a tool they call “Gate keeper” which can push revs to a different groups of Facebook users, ex. only users located in a single country, only 1% of Facebook users etc.,
• Core Release Engineers can rate programmers (give them stars) so then they can estimate the risk of pushing his/her code,
• they compile PHP code using HipHop, Facebook compiled binary is 1GB and it compiles 10 minutes (!),
• they push that binary across block using BitTorrent, it spreads in 50 minutes.

We’re recruting Facebook Developers using… Facebook app

We decided to hire Facebook Developers to help us working on our social media applications platform Megafoni. But writing a job offer and publishing it is so 2010 ;). As our project is part of Facebook enviroment in which developers live too, we created a Facebook application named: Megafoni.pl rekrutuje! in which instead of just informing about the offer, we actually do initial testing of candidates!

I’m so excited to see if this form of attracting candidates to work in your company works.

USRP: Unboxing and putting it together

Fake votes in Facebook contests

Contests which include voting part are probably the most popular Facebook promotions. Developers should be aware of the fact that the more valuable prizes are, the more application is vulnerable to fake votes from cheaters who are able to create hundreads of accounts to win the contest. In one of photo contests run on our social media applications platform Megafoni, we witnessed such fraud.

Photo uploaded by a cheater had 265 votes from which 95% were votes added by fake users. Cheater made about 250 fake accounts like this below:

It looks like real Facebook user’s profile but there are several things that make us assume that something’s wrong. Each and every profile shows minimum information about user: we can only see person’s name, photo, education and work while default privacy settings also allows others to see wall and friends. OK, somebody may have changed his/her privacy settings but it can’t be coincidence that 250 people who voted on this single photo in the contest made it. Interesting fact is that photos added to these profiles are mainly stock photos, what we’ve determined using TinEye.

Another thing worth mentioning are IPs of voters. Votes came from 4 groups of IPs, each group has first two octets the same. All IPs belong to one of mobile networks, which probably assigns a new IP every time a subscriber connects to the network.

Next thing we are able to check is verified field we get by Graph API. When user verifies account by mobile or credit card the field has value 1, otherwise 0. All of fake accounts, of course, wasn’t verified.

What’s sad, when we typed cheater name and surname in Google we found plenty of pages with contests’ results in which he was a winner. People who administrated them didn’t (or didn’t know how) remove fake votes and allowed him to win. So if you run or will be running contest on Facebook, follow these instructions to avoid situations in which the person who gets a prize doesn’t deserve it:

• Check if votes are added from the same IP addresses (or has the same firsts octets). Check IP owner.
• Try to check verified field by Graph API. However notice that no all users verify their accounts.
• Check if voters profiles seem fake.
• You may use captcha what can eliminate voting scripts.
• Require users to click a link in an e-mail sent to him/her in order to make vote valid.

You can also create algorithm which analizes factors above and determines which votes may be invalid. We developed one at Megafoni and made it part of our platform.