Lester Mackey, who speaks in a soft monotone, becomes audibly excited when discussing the end of the competition. Almost a decade later, but he can take you right back to the final days. By that point, Dinosaur Planet had been folded into a 30 person mega-team called The Ensemble, itself formed to compete with the super-group BellKor's Pragmatic Chaos. (Yes, most of these team names sound like titles for sci-fi novels.)
The BellKor squad included Chris Volinsky and his AT&T colleagues Robert Bell and Yehuda Koren, along with four other engineers from the United States, Austria, Canada and Israel. The global effort paid off: On June 26, 2009, BellKor's Pragmatic Chaos finally crossed the 10% finish line, triggering a 30 day window to submit a better algorithm. Mackey and his Ensemble teammates were feeling the heat.
"We had, of course, not slept for the past two days because the contest was about to end," says Mackey. "We knew we were close to BellKor's Pragmatic Chaos but we didn't know if we had passed them for a day. Days prior we had actually managed to inch ahead of them. But knowing that team, we knew they could easily strike back with something better."
Towards the end, the solutions had become almost absurdly complex. Mackey was mostly working as "The Blender," taking in all the algorithms and code produced by the various team members and trying to turn it into a final prediction. Minutes before he was supposed to submit his team's answer, Mackey received an email from his team member Peng Zhou, who alerted him to a better combination on their server. But when he wrote back to ask for the file name, he got radio silence. Unbeknownst to him at the time, Zhou had lost his internet connection.
Desperate, Mackey frantically searched through around a thousand files and found the right one, submitting it right as the last few seconds on the clock ticked away. But would it be enough to win? Or did BellKor have something else up their sleeve? There was no way of knowing.
"There was a fair amount of intrigue and backroom dealing going on," says Volinsky with a chuckle. "There turned out to be a lot of interesting espionage and trying to figure out who was working with who, who was in bed with who, who is giving so and so information."