Over the years, however, I've started to wonder whether Netflix's big decisions are truly as data driven as they are purported to be. The company does have more audience data than nearly anyone else (with the possible exception of YouTube), so it has a reason to emphasize its comparative advantage. But, when I was reporting a story, a couple of years ago, about Netflix's embrace of fandom over mass culture, I began to sense that their biggest bets always seemed ultimately driven by faith in a particular cult creator, like David Fincher ("House of Cards"), Jenji Leslie Kohan ("Orange is the New Black"), Ricky Gervais ("Derek"), John Fusco ("Marco Polo"), or Mitchell Hurwitz ("Arrested Development"). And, while Netflix does not release its viewership numbers, some of the company's programming, like "Marco Polo," hasn't seemed to generate the same audience excitement as, say, "House of Cards." In short, I do think that there is a sophisticated algorithm at work here -- but I think his name is Ted Sarandos.
I presented Sarandos with this theory at a Sundance panel called "How I Learned to Stop Worrying and Trust the Algorithm," moderated by Jason Hirschhorn, formerly of MySpace. Sarandos, very agreeably, wobbled a bit. "It is important to know which data to ignore," he conceded, before saying, at the end, "In practice, its probably a seventy-thirty mix." But which is the seventy and which is the thirty? "Seventy is the data, and thirty is judgment," he told me later. Then he paused, and said, "But the thirty needs to be on top, if that makes sense."
Some of you will know that Average is Over contains an extensive discussion of "freestyle chess," where humans can use any and all tools available -- most of all computers and computer programs -- to play the best chess game possible. The book also notes that "man plus computer" is a stronger player than "computer alone," at least provided the human knows what he is doing. You will find a similar claim from Brynjolfsson and McAfee.
Computer chess expert Kenneth W. Regan has compiled extensive data on this question, and you will see that a striking percentage of the best or most accurate chess games of all time have been played by man-machine pairs. Ken's explanations are a bit dense for those who don't already know chess, computer chess, Freestyle and its lingo, but yes that is what he finds, click on the links in his link for confirmation. In this list for instance the Freestyle teams do very very well.
I wonder what the human/cyborg split is at Buzzfeed or Facebook? Or at food companies like McDonald's or Kraft? Or at Goldman Sachs?
In terms of comparison, Magnus Carlsen, the world's current #1 and owner of the highest ranking ever, is 2-1-4 at the same tournament. Which is pretty typical; the best players draw a lot. Over his career, Carlsen has drawn almost 50% of the time and Caruana about 40%.
The modern times of chess have a new king, king Fabiano Caruana. One has to look back to 1968 where in Wijk Aan Zee the legendary Korchnoi started with 8,0/8. The times now are so different and the competition so fierce that already Fabiano's success can be proclaimed as the most memorable streak in the history of chess.
Along the way, Caruana has beaten Carlsen (#1), Levon Aronian (#2), Maxime Vachier-Lagrave (#9) twice, Hikaru Nakamura (#7), and Veselin Topalov (#6) twice. If you look at the unofficial live chess ratings, you'll see he has moved into the #2 position in the world, jumping a whopping 34.1 points in rating. He also owns the fourth highest rating in history, behind Carlsen, Kasparov, and Aronian. Caruana plays Carlsen again today, starting from the more advantageous white position. (via @tylercowen)
It's a well-known fact that White has a small advantage at the beginning of the game. To maintain this advantage, White should press their advantage to take over the middle of the board as quickly as possible. The most popular first White moves from 1850-2014 are shown below. Note that all of these are fairly aggressive openings that build toward control of the middle of the board.
In 1850, White openings were fairly homogeneous: Most chess experts played King's Pawn. Chess players didn't begin to explore variants of the King's Pawn in earnest until the 1890s, when Queen's Pawn (moving a Pawn to d4) started to replace King's Pawn in some player's repertoires. The 1920s saw another burst of innovation with the rising popularity of the Zukertort Opening (moving the Knight to f3) and the English Opening (moving a Pawn to c4), which completed the set of staple first-turn openings that are really ever used nowadays.
"Tell me about your game," Spiegel said. Sebastian flopped into the chair and handed her his notepad, where he'd recorded all the moves for both players in the game.
Sebastian explained that the other guy was simply better. "He had good skills," he said. "Good strategies."
And this is the point where many of us would simply say something along the lines of "did you do your best?," in which case the likely response is "Yes." Everyone is at least let off the hook. The teacher for ensuring students try their best, the student for having lost to someone better. Spiegel did not take this approach.
The most critical missing piece, Randolph explained as we sat in his office last fall, is character -- those essential traits of mind and habit that were drilled into him at boarding school in England and that also have deep roots in American history. "Whether it's the pioneer in the Conestoga wagon or someone coming here in the 1920s from southern Italy, there was this idea in America that if you worked hard and you showed real grit, that you could be successful," he said. "Strangely, we've now forgotten that. People who have an easy time of things, who get 800s on their SAT's, I worry that those people get feedback that everything they're doing is great. And I think as a result, we are actually setting them up for long-term failure. When that person suddenly has to face up to a difficult moment, then I think they're screwed, to be honest. I don't think they've grown the capacities to be able to handle that."
The best chess player in history, 23-year-old Norwegian Magnus Carlsen, has released an iOS app where you can play simulated games against Carlsen at various stages of his career, from age 5 up to the present. The Telegraph has the details.
Anyone who wants to find out more about his playing style can do so with Mr Carlsen's new app, which allows users to play him at the different levels he has achieved since the age of five.
The app is built on hundreds of thousands of different positions from Mr Carlsen's games, be they classical, rapid or blitz, to determine what moves he would make at those ages.
The aim is to promote chess among as many people as possible to make the sport more popular and accessible.
"The good thing is that you can play me at any age. At age five, anyone has a chance to beat me," Mr Carlsen said.
So what is it like for Mr Carlsen to play against his younger self?
"He is really tricky," the champion said. "Even Magnus at 11 years old was a very gifted tactician. A while ago I played as a test Magnus [aged] 14. I outplayed him at some point positionally. And just boom, boom, he tricked me tactically.
"But he makes mistakes as well, so I just have to be patient."
I played chess with my dad growing up, but actually learning how to play (studying openings, notation, etc.) seemed daunting and at cross purposes with what I liked about the game. So I stopped playing sometime in college and never really picked it up again. But I've maintained a non-playing interest in the game and have even been playing a little bit recently again, teaching my son how to play. The other day I ran across mate-in-one puzzles (iOS app), which seem more my speed.
The tl;dr of this training plan is, play a lot, analyze your games, and primarily study tactics. Your knowledge of openings, endgame, middlegame, etc. will come from analyzing your games and going over grandmaster games. Only study one of those specific topics if it is clear you are specifically losing because of that topic.
The Chess World Championship is currently underway in Chennai, India. Through nine games, challenger and world #1 Magnus Carlsen is leading reigning world champ and world #8 Viswanathan Anand by the score of 6-3. It seems as though Carlsen's nettlesomeness is contributing to his good fortunes.
Second, Carlsen is demonstrating one of his most feared qualities, namely his "nettlesomeness," to use a term coined for this purpose by Ken Regan. Using computer analysis, you can measure which players do the most to cause their opponents to make mistakes. Carlsen has the highest nettlesomeness score by this metric, because his creative moves pressure the other player and open up a lot of room for mistakes. In contrast, a player such as Kramnik plays a high percentage of very accurate moves, and of course he is very strong, but those moves are in some way calmer and they are less likely to induce mistakes in response.
Or perhaps Carlsen's just inspired by the lovely chess set they're using? Either way, he needs just one draw in the remaining three games to win the Championship. Or putting it another way, Anand has to win all three of the remaining games to retain the title.
White moves first, but then Black gets to move twice. Then White gets to move three times in a row, then Black four times in a row, then White five times in a row, and so on, with continuing escalation as the game proceeds.
I'm overwhelmed to be in the same room as these men. I played through their games (well, except the younger ones) as a teenager, developing a love or dislike of their styles, and scratching my head at their depth. The skill gap in chess is remarkable: these Grandmasters would demolish someone who would easily beat someone who would wipe me off the board. Amid my admiration, I feel a vertiginous impulse: I could leap out of my seat, scatter the pieces, and make history as the world's first chess streaker. The temptation soon fades.
Each player aligns the pieces, although the boards are already laid out in pristine formation. It's a curious habit I recognise from my own experience. It helps to get your hands on the tools of your trade, to feel they're yours.
As chess increased in popularity across Europe in the 1800s, the proliferation in the variety of chess sets caused confusion amongst competitors, especially those hailing from different countries. The English typically used Barleycorn sets:
English saw a different design for every chess club: St. George sets with their appearance of stacked disks, Dublin sets with more rounded middles, and Northern Uprights with columns instead, as well as elaborate, easily tipped Barleycorn sets. Germany had delicate Selenus sets, beautiful beyond belief, but fragile, tippable, and problematic for play. To tell which piece is which on some of these sets one must count the stacked crown. France saw elegant Regence style sets with some of the most confusing signatures in history. As in the English sets, queen's were represented by orbs. The king's floral crown closely resembles the modern Staunton signature for the queen. Knights were always taller than bishops the old French sets. Bishops were represented as fools, not clergymen, and therefore lacked the signature miter. What was worse, the knights in these sets were sometimes simple turned designs, not the recognizable horse's head. This lead to common confusion as to which minor piece was which. The confusion of antique French knights and bishops is still a common problem today.
Then in the 1849, Nathaniel Cook designed and John Jaques began to sell a set that eventually came to be called the Staunton chess set:
Howard Staunton was regarded as the top chess player of his era and organized the first international chess tournament in 1851. Staunton endorsed the set and it soon became the standard in chess competitions and, later, the official standard of the World Chess Federation. The most recent iteration of the official Staunton set is Daniel Weil's design for World Chess:
If you're interested in learning more, Jimmy Stamp has a nice piece about the design of the original Staunton set and Weil's update at Smithsonian magazine.
Grantland has a story about a chess cheating scandal with an interesting section on the the history of chess-playing computers.
The Virginia scandal involved the opposite ruse, in which a machine surreptitiously called the shots for a player. The chess engines this scheme centered on are relatively new: Computers only surpassed humans at the chessboard during young Smiley's lifetime. Scientists had an easier time designing digital brains that could produce atom bombs or navigate lunar landings than they did fashioning a machine that could play chess worth a darn. Plainly, until relatively recently, chess was too complicated for computers. An analysis of chess's complicatedness in Wired determined that the number of possible positions in an average 40-move game is 10 to the 128th power, a sum "vastly larger than the number of atoms in the known universe."
In 1966, MIT brainiacs entered MAC HACK VI, a computer program they'd devised, into the Massachusetts Amateur Chess Championship, making it the first computer program ever to enter a tournament. It drew just one match and lost four.
But by 2007, a chess engine called Rybka was routinely shutting out grandmasters even when spotting the humans a pawn and taking black, thereby letting humans go first, the more statistically desirable position. Computers have gotten noticeably better since then; humans haven't.
The man-machine war in chess is no longer contested: "Computers are better than us," says USCF president Ruth Haring.
Stanley Kubrick didn't give long interviews...or didn't like giving them anyway. But Jeremy Bernstein convinced him to sit down for one, perhaps because Kubrick was a huge chess nerd and Bernstein played chess seriously. So the two of them did this hour-long interview in 1965 that resulted in this New Yorker piece about his life, films, and the then in-production 2001.
During our conversation, I happened to mention that I had just been in Washington Square Park playing chess. He asked me who I had been playing with, and I described the Master. Kubrick recognized him immediately. I had been playing a good deal with the Master, and my game had improved to the point where I was almost breaking even with him, so I was a little stunned to learn that Kubrick had played the Master on occasion, and that in his view the Master was a potzer. Kubrick went on to say that he loved playing chess, and added, "How about a little game right now?" By pleading another appointment, I managed to stave off the challenge.
Deep Blue was able to play chess well because the game is perfectly logical, with fairly simple rules; it can be reduced easily to math, which computers handle superbly. But the rules of language are much trickier. At the time, the very best question-answering systems -- some created by software firms, some by university researchers -- could sort through news articles on their own and answer questions about the content, but they understood only questions stated in very simple language ("What is the capital of Russia?"); in government-run competitions, the top systems answered correctly only about 70 percent of the time, and many were far worse. "Jeopardy!" with its witty, punning questions, seemed beyond their capabilities. What's more, winning on "Jeopardy!" requires finding an answer in a few seconds. The top question-answering machines often spent longer, even entire minutes, doing the same thing.
The match between Anand and Topalov was hard fought, partly because Topalov invoked a rule for the contest that forbids the players from offering draws to each other. The rule, named after the city where the match was being played, insured that there would be no short draws. As the match wore on and fatigue took a toll, both players began to make mistakes with greater frequency.
"Anand" was briefly a global Trending Topic on Twitter this afternoon, which was unexpected and nice.
All during the filming of 2001 we played chess whenever I was in London and every fifth game I did something unusual. Finally we reached the 25th game and it was agreed that this would decide the matter. Well into the game he made a move that I was sure was a loser. He even clutched his stomach to show how upset he was. But it was a trap and I was promptly clobbered. "You didn't know I could act too," he remarked.
Carlsen: I have no idea. I wouldn't want to know it anyway. It might turn out to be a nasty surprise.
SPIEGEL: Why? You are 19 years old and ranked the number one chess player in the world. You must be incredibly clever.
Carlsen: And that's precisely what would be terrible. Of course it is important for a chess player to be able to concentrate well, but being too intelligent can also be a burden. It can get in your way. I am convinced that the reason the Englishman John Nunn never became world champion is that he is too clever for that.
SPIEGEL: How that?
Carlsen: At the age of 15, Nunn started studying mathematics in Oxford; he was the youngest student in the last 500 years, and at 23 he did a PhD in algebraic topology. He has so incredibly much in his head. Simply too much. His enormous powers of understanding and his constant thirst for knowledge distracted him from chess.
SPIEGEL: Things are different in your case?
Carlsen: Right. I am a totally normal guy. My father is considerably more intelligent than I am.
His comparison of his abilities with Garry Kasparov's later in the interview is interesting as well.
The heavy use of computer analysis has pushed the game itself in new directions. The machine doesn't care about style or patterns or hundreds of years of established theory. It counts up the values of the chess pieces, analyzes a few billion moves, and counts them up again. (A computer translates each piece and each positional factor into a value in order to reduce the game to numbers it can crunch.) It is entirely free of prejudice and doctrine and this has contributed to the development of players who are almost as free of dogma as the machines with which they train. Increasingly, a move isn't good or bad because it looks that way or because it hasn't been done that way before. It's simply good if it works and bad if it doesn't. Although we still require a strong measure of intuition and logic to play well, humans today are starting to play more like computers.
The section about people using computers *during* matches is particularly interesting.
The idea behind Democratic Chess is that the pieces, in collaboration with the players, decide where to move. Each piece communicates with the others through built-in cameras, microphones, and speakers.
Democratic Chess is Chess game where each figure is made of an IP-WLan-network camera each capable of looking arround, listening and talking to the other figures as well as the 2 real person players. With this technology there are many different ways how to play the Game, the next move can be decided in a democratic way among the Figures or they are allowed to discuss with the players and each other the next moves, but at the End the 2 player make the moves.
Perhaps Democratic Chess will crush the evil dictatorial tyranny of the chess player once and for all. (via buzzfeed)
This was the beauty of Bobby Fischer's mind, even then. The boy made very clean, simple lines out of very complex problems, and when the trap was sprung, his style of chess became so transparent you could instantly recognize its brilliance: efficient, organic, wildly responsive and creative.
Blindfold chess is playing chess without a board or pieces...you've got to remember where everything is in your head. The world record holder played 45 games of blindfold chess simultaneously. More at Wikipedia. (via panopticist)
Oh, in other #1 news, Serena Williams will be the new #1 in women's tennis after beating Jelena Jankovic in the final of the US Open. On the men's side, world #1 Rafael Nadal lost in the semis to Andy Murray but won't lose the top spot in the rankings.
Never mind that I'd declined his offer of a lesson, Murphy had gone ahead and transformed our discussion into a formal chess tutorial to which a ticking meter was attached. When the talk wound down, he presented me with a verbal invoice for $20, his standard teaching rate. The chess instruction aside, the $20 I spent taught me an even more memorable lesson about Murphy: When you are in his company, there is often a second, invisible chess game taking place, one that can easily conclude with Murphy's rooks advancing on your wallet.
In recent years, Putin has insured that nearly all power in Russia is Presidential. The legislature, the State Duma, is only marginally more independent than the Supreme Soviet was under Leonid Brezhnev. The governors of Russia's more than eighty regions are no longer elected, as they were under Yeltsin; since a Presidential decree in 2004, they have all been appointed by the Kremlin. Putin even appoints the mayors of Moscow and St. Petersburg. The federal television networks, by far the main instrument of news and information in Russia, are neo-Soviet in their absolute obeisance to Kremlin power.
Mashup sport: chessboxing. "The basic idea in chessboxing is to combine the no.1 thinking sport and the no.1 fighting sport into a hybrid that demands the most of its competitors - both mentally and physically. In a chessboxing fight two opponents play alternating rounds of chess and boxing. The contest starts with a round of chess, followed by a boxing round, followed by another round of chess and so on." More from the LA Times and the Guardian.