Palantir Technologies

Dans un billet d'il y a deux jours, je me moquais gentiment des systèmes de recommandations (mais je reconnais les performances des ces systèmes dans le cas général). Entre temps, je suis tombé par hasard sur un article de Shobian Gorman titré How Team of Geeks Cracked Spy Trade dans le Wall Street Journal daté du 4 septembre. Il présente Palantir Technologies, un éditeur de logiciel visant à faciliter la recherche de terroristes par une meilleur utilisation des données connues par les analystes.

The software's main advance is a user-friendly search tool that can scan multiple data sources at once, something previous search tools couldn't do. That means an analyst who is following a tip about a planned terror attack, for example, can more quickly and easily unearth connections among suspects, money transfers, phone calls and previous attacks around the globe.

J'aime l'approche adoptée :

[Palantir] rejected advice to hire retired generals to curry favor with the agencies and hired young government analysts frustrated by working with slow-footed technology. The company's founders knew little about intelligence gathering when they started out. Instead, they went on a fact-finding mission, working with analysts to build the product from scratch.

La société a été financée par Peter Thiel un des cofondateurs de PayPal. C'est lui qui a eu l'idée de base d'appliquer à la recherche des terroristes des méthodes utilisées par PayPal pour identifier les fraudes sur le site.

PayPal's software could make connections between fraudulent payments that on the surface seemed unrelated. By following such leads, PayPal was able to identify suspect customers and uncover cybercrime networks. The company saw a tenfold decrease in fraud losses after it launched the software, while many competitors struggled to beat back cheaters.

Au passage, on note là un exemple de recherches théoriques menées par une société privée et appliquées ensuite à des champs bien plus vastes que cette entreprise.

Spy agencies like the CIA and military intelligence organizations have hundreds of databases each, most of which aren't linked up. A single database might contain reports from field agents or lists of known terrorists or companies thought to be financing terrorism. To conduct an investigation, analysts have to query individual databases separately, then try to make sense of the data -- frequently with pen and paper.
With many of the existing search tools, analysts also can't access some files on terrorist suspects or other threats because a bit of data in the file is classified at a level higher than they are allowed to see. That is a problem, because making connections among new clues and existing data is a key to foiling terrorist plots. Among the missed opportunities cited by post-9/11 investigations were the failure to see that five of the 19 hijackers used the same phone number as ringleader Mohammad Atta to book their airline tickets, two used the same frequent-flier number, and five used two common addresses to make their reservations.

Le problème actuel des analystes est clairement expliqué ici et on voit bien l'apport des nouvelles méthodes proposées par Palantir.

En même temps, je ne peux m'empêcher d'être inquiet face à ces progrès du datamining qui sont aussi potentiellement des menaces pour les libertés civiles. Le même outils aidera les dictatures telles que la Chine et la Syrie à repérer les dissidents.

Espérons que ces techniques feront plus de Bien que de Mal.