Does Palantir See Too Much?

Does Palantir See Too Much?

The tech giant helps governments and law enforcement decipher vast amounts of data — to mysterious and, some say, dangerous ends.

By Michael Steinberger

On a bright Tuesday afternoon in Paris last fall, Alex Karp was doing tai chi in the Luxembourg Gardens. He wore blue Nike sweatpants, a blue polo shirt, orange socks, charcoal-gray sneakers and white-framed sunglasses with red accents that inevitably drew attention to his most distinctive feature, a tangle of salt-and-pepper hair rising skyward from his head.

Under a canopy of chestnut trees, Karp executed a series of elegant tai chi and qigong moves, shifting the pebbles and dirt gently under his feet as he twisted and turned. A group of teenagers watched in amusement. After 10 minutes or so, Karp walked to a nearby bench, where one of his bodyguards had placed a cooler and what looked like an instrument case. The cooler held several bottles of the nonalcoholic German beer that Karp drinks (he would crack one open on the way out of the park). The case contained a wooden sword, which he needed for the next part of his routine. “I brought a real sword the last time I was here, but the police stopped me,” he said matter of factly as he began slashing the air with the sword.

Those gendarmes evidently didn’t know that Karp, far from being a public menace, was the chief executive of an American company whose software has been deployed on behalf of public safety in France. The company, Palantir Technologies, is named after the seeing stones in J.R.R. Tolkien’s “The Lord of the Rings.” Its two primary software programs, Gotham and Foundry, gather and process vast quantities of data in order to identify connections, patterns and trends that might elude human analysts. The stated goal of all this “data integration” is to help organizations make better decisions, and many of Palantir’s customers consider its technology to be transformative. Karp claims a loftier ambition, however. “We built our company to support the West,” he says. To that end, Palantir says it does not do business in countries that it considers adversarial to the U.S. and its allies, namely China and Russia. In the company’s early days, Palantir employees, invoking Tolkien, described their mission as “saving the shire.”

The brainchild of Karp’s friend and law-school classmate Peter Thiel, Palantir was founded in 2003. It was seeded in part by In-Q-Tel, the C.I.A.’s venture-capital arm, and the C.I.A. remains a client. Palantir’s technology is rumored to have been used to track down Osama bin Laden — a claim that has never been verified but one that has conferred an enduring mystique on the company. These days, Palantir is used for counterterrorism by a number of Western governments. French intelligence turned to Palantir following the November 2015 terror attacks in Paris. Karp claims that Palantir has helped thwart several attacks, including one or two that he says could have had seismic political consequences. “I believe that Western civilization has rested on our somewhat small shoulders a couple of times in the last 15 years,” he told me in Paris, where he was hosting a conference for Palantir’s corporate clients.

A few months later, the world was being menaced by a novel coronavirus, and Palantir quickly joined that battle against Covid-19: By April, according to the company, approximately a dozen countries were using its technology to track and contain the virus. The speed with which Palantir transitioned to pandemic response ostensibly underscores the flexibility of its software, which can be put to any number of tasks. The U.S. Army uses it for logistics, among other things. The investment bank Credit Suisse uses it to guard against money laundering. The pharmaceutical company Merck, in Germany, uses it to expedite the development of new drugs. Ferrari Scuderia uses it to try to make its Formula 1 cars faster. To Palantirians, as some call themselves, these myriad applications are just further proof that many problems are data-integration problems.

Yet Palantir’s work on the coronavirus has also highlighted the mistrust that trails the company. In Europe, it is viewed with suspicion because of the C.I.A. connection. But the main source of apprehension is simply the nature of Palantir’s work. Although Palantir claims it does not store or sell client data and has incorporated into its software what it insists are robust privacy controls, those who worry about the sanctity of personal information see Palantir as a particularly malignant avatar of the Big Data revolution. Karp himself doesn’t deny the risk. “Every technology is dangerous,” he says, “including ours.” The fact that the health records of millions of people are now being funneled through Palantir’s software has only added to the unease.

That’s especially true in the United States, where the Department of Health and Human Services is using Palantir’s software to analyze virus-related data. Palantir’s work with H.H.S. has become bound up in the biggest controversy that the company has faced, over its relationship with United States Immigration and Customs Enforcement. Progressive activists and members of Congress have expressed fear that the information collected by H.H.S. could be used by the Trump administration to expand its immigration crackdown, in which Palantir’s technology has played a part. And the fact that Palantir was awarded a pair of no-bid contracts valued at nearly $25 million by H.H.S. has amplified concerns that it has benefited from Thiel’s support of President Trump. Thiel was one of his most prominent backers in 2016, even speaking at the Republican National Convention.

Palantir’s perceived links to the president have made it an object of suspicion among liberals, which frustrates Karp. In contrast to Thiel, the 53-year-old Karp is a self-described “progressive warrior” who says he voted for Hillary Clinton and who has expressed antipathy for Trump. His greatest fear, he says, is the rise of fascism. Although Karp’s political views are widely shared in Silicon Valley, he is one of the tech industry’s unlikeliest chief executives. He co-founded Palantir with no background in computer science or business. Instead, he holds a law degree from Stanford University and a doctorate in social theory from Goethe University in Frankfurt, where for a time his thesis adviser was Jürgen Habermas, possibly Europe’s most celebrated living social philosopher. On the corporate scene, Karp is a sui generis figure, a fact vividly on display that autumn afternoon in the Luxembourg Gardens.

Until recently, it could be argued that his intellectual pedigree and political leanings were a kind of shield for Palantir, deflecting criticism of its work — or at least keeping critics off balance. But fairly or not, Palantir has come to be regarded as an enabler and prime beneficiary of Trump’s presidency, which has rendered the company toxic in the eyes of many progressives. In response to the criticism of Palantir’s relationship with ICE, Karp has attacked the tech industry over what he sees as its insufficient patriotism. Palantir recently relocated its headquarters from Palo Alto to Denver, a move that seemed partly rooted in the contempt that Karp and Thiel have for Silicon Valley. The company, which has yet to turn a profit, went public last month amid concerns that its prospects in Washington could be diminished under a Biden administration. Palantir says that its software solves “the world’s hardest problems.” Removing the stain of Trumpism may prove to be an especially hard one.

Speaking at a tech conference in 2010, Eric Schmidt, Google’s chief executive at the time, made a startling observation. “There were five exabytes of information created by the entire world between the dawn of civilization and 2003. Now that same amount of information is created every two days,” he said. (An exabyte is equivalent to one billion gigabytes.) It was perhaps a slightly exaggerated claim in the service of an indisputable fact: Humanity is now awash in data. The premise of Big Data is that all of this information can yield powerful insights. But the difficulty is harnessing the data, which is where Palantir comes in. Although Palantir has glamorous clients and offices in upscale locales (the Marais in Paris, Soho Square in London, the Georgetown section of Washington), in tech circles, data integration is not considered particularly sexy. “It’s plumbing work, basically,” Louis Mosley, who runs Palantir’s London office, told me with a smile.

He was being modest. What Palantir does is a little more complex than unclogging a toilet. Essentially, Palantir’s software synthesizes the data that an organization collects. It could be five or six types of data; it could be hundreds. The challenge is that each type of information — phone numbers, trading records, tax returns, photos, text messages — is often formatted differently from the others and siloed in separate databases. Building virtual pipelines, Palantir engineers merge all the information into a single platform. They work quickly. According to Jose Arrieta, who was H.H.S.’s chief information officer until two months ago, Palantir merged around two billion data elements related to the Covid-19 outbreak in less than three weeks. Once the data has been integrated, it can be presented in the form of tables, graphs, timelines, heat maps, artificial-intelligence models, histograms, spider diagrams and geospatial analysis. It is a digital panopticon, and having sat through several Palantir demos, I can report that the interface is impressive — the search results are strikingly elegant and easy to understand.

Those appealing visuals were conceived in order to hunt and kill terrorists. In 1998, Thiel co-founded PayPal, then served as its chief executive from 2000 until it was acquired by eBay in 2002. Not long after 9/11 — Thiel can’t recall exactly when — it occurred to him that PayPal’s anti-fraud algorithms could possibly help the U.S. government combat terrorism. In 2003, Thiel asked a trio of software engineers, including two from PayPal, to create a prototype. His intuition plus their coding gave rise to Palantir. While Thiel provided most of the early money, the start-up secured an estimated $2 million from In-Q-Tel, a venture-capital firm that finances the development of technologies that can help the C.I.A.

Karp says the real value of the In-Q-Tel investment was that it gave Palantir access to the C.I.A. analysts who were its intended clients. According to Palantir, every two weeks, Aki Jain, one of the first engineers hired by Thiel, and Stephen Cohen, an engineer who had worked at Thiel’s hedge fund, Clarion Capital Partners, traveled from Palo Alto to Langley with an updated version of the software program. (Cohen recalls the C.I.A. guys’ referring to him as “Two Weeks.”) The C.I.A. analysts would test it out and offer feedback, and then Cohen and Jain would fly back to California to tweak it. Jain estimates that from 2005 to 2009, he and Cohen made around 200 trips to Virginia. The iterative approach became standard practice for Palantir — even now, it embeds what it calls “forward-deployed engineers” with clients to customize the software to their needs, which has led some observers to conclude that Palantir is as much a consultancy as it is a software maker.

Although Palantir is often depicted as a kind of omnipotent force, it is actually quite small, with around 2,400 employees. By contrast, Facebook, which seems to vie with Palantir for the worst headlines these days, has more than 50,000. (Thiel was the first outside investor in Facebook and remains a member of its board.) And while Palantir’s reach feels tentacular, the prospectus that it filed with the Securities and Exchange Commission before going public revealed that it has just 125 customers, a number that surprised some observers and raised questions about the company’s growth prospects. In mid-October, Palantir stock was trading around $10 per share, and its market capitalization was nearly $16 billion.

Palantir is pricey — customers pay $10 million to $100 million annually — and not everyone is enamored of the product. Home Depot, Hershey, Coca-Cola and American Express all dropped Palantir after using it. Even within the intelligence community, there seem to be mixed opinions. Three years ago, BuzzFeed obtained a leaked video in which Karp told Palantir employees that the company’s relationship with the National Security Agency had ended. Several former C.I.A. analysts told me that they and their colleagues were underwhelmed by Palantir. But the C.I.A. is a big place, and others who worked there extolled it.

Some clients seem to believe Palantir’s software is essential. One is the aerospace manufacturer Airbus, which hired Palantir in 2016 when it was ramping up production of its new A350 jet. Marc Fontaine, who until recently was Airbus’s digital-transformation officer, told me that when you go from a single plane on the assembly line to 10, “the complexity increases exponentially, and it kills you.” Missing parts, faulty parts, production mistakes, communications glitches — those and other problems inevitably slow down the assembly process and cause millions of dollars in cost overruns. They can also result in penalties and damages that have to be paid to airlines awaiting delivery.

In 2016, five Palantir engineers embedded in the Airbus factory in Toulouse, France. Using Foundry, Palantir’s commercial application — Gotham, its other flagship software program, is for national security and defense — they merged 25 data silos related to production of the A350 and integrated more than 400 sets of data. Palantir produced results immediately. Before it came on board, Fontaine says, it took an average of 24 days to fix production mistakes; Palantir helped cut that to 17. Airbus realized several hundred million dollars in cost savings.

These days, around 15,000 Airbus employees use Palantir, and its software has essentially wired the entire Airbus ecosystem through a venture called Skywise, which collects and analyzes data from around 130 airlines worldwide. The information is used for everything from improving on-time performance to preventive maintenance. Fontaine says that Airbus was always open to using other data-analytics tools, but “we didn’t find anything equivalent at the time to Palantir.” Its software, he says, “has unique capability.” His former boss, Tom Enders, who was Airbus’s chief executive from 2012 to 2019, echoes that praise. He calls bringing in Palantir “one of the best decisions of my career.”

‘‘I still can’t believe I haven’t been shot and pushed out the window,” Karp told me. We were in Palantir’s New York office, located in the Meatpacking district. He wasn’t being literal, despite the office’s bulletproof windows and the bodyguards hovering nearby. Rather, he meant the feeling of inevitable doom that has plagued him since childhood. Karp grew up in the Philadelphia area. His parents are Dr. Robert Karp, a clinical pediatrician, and Leah Jaynes Karp, an artist. His father is Jewish; his mother is African-American. (Karp has a brother and two stepsiblings.) He told me that his parents were “hippies” and that he spent a lot of time as a kid at political protests. He intuited from a young age that his background made him vulnerable, he said. “You’re a racially amorphous, far-left Jewish kid who’s also dyslexic — would you not come up with the idea that you’re [expletive]?” Although he was now the head of a major corporation, neither time nor success had diminished the anxiety. If the far right came to power, he said, he would certainly be among its victims. “Who’s the first person who is going to get hung? You make a list, and I will show you who they get first. It’s me. There’s not a box I don’t check.” His fear, he said, “propels a lot of the decisions for this company.”

Given the political milieu in which he was raised, Haverford College, a school with Quaker roots near Philadelphia with a robust tradition of dissent (it was a hive of antiwar activity in the 1960s and 1970s), was a natural fit for Karp. We happened to be classmates there, but despite the college’s small size (currently around 1,200 students), he and I somehow never exchanged a word in four years. Karp was, by his own admission, somewhat introverted and also very studious. The library didn’t see much of me, which may go some way to explaining why he ended up a billionaire and I did not. Over conversations in New York, Washington, Paris and Vermont, we found that we had a lot to talk about, although I can’t say it is regrettable that we never connected at Haverford; I am not sure that 20-year-old me would have fully appreciated his bracing intelligence, and I suspect that my talent for procuring beer and organizing Roman-themed parties would have been of little use to him.

After graduating, Karp went to Stanford Law School, which he hated — “the worst three years of my adult life.” He says he knew within a week of enrolling that he had made a mistake. In his view, Stanford was just a glorified trade school; his classmates were mainly animated by a desire to land prestigious jobs, and the intellectual discourse was “highly performative,” as he puts it. What made Stanford bearable was his unlikely friendship with Thiel, a classmate. They bonded over their shared disdain for law school and a love of political debate. Thiel had already achieved some prominence for his libertarian views — as a Stanford undergraduate, he had helped found the right-leaning Stanford Review — and he and Karp spent much of their free time interrogating each other’s positions. “We argued like feral animals,” Karp recalled. According to Thiel, their conversations generally took place late at night in the law-school dorm. “It sounds too self-aggrandizing, but I think we were both genuinely interested in ideas,” he says. “He was more the socialist, I was more the capitalist. He was always talking about Marxist theories of alienated labor and how this was true

Karp didn’t even stick around for his Stanford graduation: As soon as classes ended, he left for Frankfurt to begin studying German. His aim was to earn a doctorate in Germany, an ambition kindled mainly by the fact that most of the writers and thinkers he was drawn to were German. After six months, he had mastered enough of the language — despite his dyslexia — to gain admission to Goethe University in Frankfurt. Having Jürgen Habermas as his Ph.D. thesis adviser was a big deal. Habermas was affiliated with the university’s Institute for Social Research, which had given rise to the so-called Frankfurt School, a neo-Marxist movement renowned for its critique of capitalism and culture. In Karp’s words, “If you can get Habermas to work with you for even two minutes, you can be a tenured professor at Columbia.”

But Karp says he had a falling out with Habermas over his dissertation topic and ended up switching advisers. When I first asked him to describe his thesis, which he wrote in German, he said that it “rebuilt the Parsonian framework to account for the somewhat irrational philosophy of Adorno, basically.” When I later asked for an explanation that I could perhaps understand, he told me that it was about the German writer Martin Walser’s controversial 1998 speech on the limits of wartime guilt and “a parochial form of fascism that occurs by purposely saying things that are incorrect in speech.” (“Parsonian” is a reference to the American sociologist Talcott Parsons; Theodor W. Adorno was a German philosopher and sociologist.) Karp said that although his collaboration with Habermas ended prematurely, it was clarifying. He realized that, however gifted a scholar he might have been, he could never attain the stature of Habermas. “Working with Habermas showed me that I couldn’t be him and didn’t want to be him,” he says.

While his second advanced degree also failed to yield a career path, it had an unexpected dividend: He developed a deep affinity for Germany. “I went for intellectual reasons,” he says. “The reason I stayed was emotional.” He found that he was good at what he calls “German conceptual thinking” but also felt a sense of belonging in Germany — that he fit in even as he recognized that his Jewishness would always set him apart. He still feels that way. “I have a second home, and it’s called Germany and the German-speaking world,” he told me. “I’m more naturally accepted there than anywhere else in the world.” The years that he spent in Germany are the touchstone of his life. “I only made two good decisions as an adult: going to Germany and starting Palantir,” he said. “Everything else was, I wouldn’t call it a mistake, but it was either preparation for these two decisions or a mistake.”

After finishing his dissertation, Karp founded a money-management firm. His goal was to accumulate a $250,000 nest egg and to settle in Berlin, where he planned to live as a highbrow dilettante, combining intellectual pursuits with various “forms of debauchery.” But then Thiel reached out to him. Thiel thought that Palantir might be a tough sell to potential clients, at least initially — there would be skepticism about the software, as well as bureaucratic resistance — and that the fledgling company needed a persuasive frontman with a sophisticated mind. “I am not sure that Alex was the perfect person for it, but he was by far the best person I knew,” Thiel told me. “You needed someone who was smart, scrappy, who — I think he has a terrific sense for people. I think he’s incredibly tenacious.”

Still, Karp was not an obvious choice to run a tech firm or any company, for that matter. Even though he has now been Palantir’s chief executive for 17 years and is a celebrity at Davos and other elite gatherings, in some ways he still seems ill suited to the task, a point he readily concedes. He is a lifelong bachelor who is drawn to solitary pursuits — his chief pastime is cross-country skiing (he will do 40 to 60 kilometers a day when time and weather permit). He doesn’t like to give speeches or do interviews, and you will never see him prowling a stage in the way that, say, Steve Jobs once did. Backslapping and small talk are not his thing, either. “Most businesspeople have a slight politician inside them,” Karp said. “I don’t have that inside me.”

He said it would be helpful to Palantir if he were more “norm-conforming.” I asked what norm-conforming looked like. “The way I see your life,” he said. (I had filled him in on the details during a previous conversation.) I was a little disappointed to learn that my conventionality — wife, kids, dog — was so obvious, but I kept that to myself and asked if he thought he could eventually marry. Karp shook his head. “I fantasized about being norm-conforming, but I don’t know how to do that,” he said. “I just don’t know how to do that, I don’t know how it works, I wouldn’t know how to be not transgressive. I try. I try really hard, really I do … but it’s not working out.” (He did tell me later that he had a girlfriend in Germany.) I suggested that his idiosyncratic personality didn’t seem to be hurting Palantir. He disagreed. “We are an enterprise company with enterprise clients,” he said. “You think it is helpful having a fluorescent praying mantis coming into their office, telling them about German philosophy? Do you think that’s helpful? I can tell you, it’s not helpful.”

On the other hand, Karp had no doubt that he was the right person to lead Palantir internally. “Once I stumbled on it,” he said, “it turned out that I was built for certain things that are really valuable, like managing very complex, sometimes difficult — highly in many cases — technical software engineers. There are just very few people in the world built for that.” Among Palantirians, “Dr. Karp,” as he is known, commands something approaching reverence. He appears to be a loyal and generous boss. I also had the impression that his employees serve a loco familia function for him, which may go some way to explaining why, before the pandemic, he spent roughly 300 days a year on the road, circulating among Palantir’s 22 offices. And if, as he insists, his distinctive manner is off-putting to clients, Karp believes it is crucial to his ability to lead Palantir. “I manage the most eclectic, creative group of 2,400 people perhaps in the world,” he told me. “You need a way you can bond, and my eccentric, nonstandard character is the bonding mechanism.”

Karp and Thiel say they had two overarching ambitions for Palantir early on. The first was to make software that could help keep the country safe from terrorism. The second was to prove that there was a technological solution to the challenge of balancing public safety and civil liberties — a “Hegelian” aspiration, as Karp puts it. Although political opposites, they both feared that personal privacy would be a casualty of the war on terrorism. When I met with Thiel at his Los Angeles office, in a conference room with a commanding view of the Hollywood Hills, he used a whiteboard to illustrate those concerns. With a black marker, he drew a graph. At the end of one axis he wrote “Dick Cheney” and at the other end he wrote “A.C.L.U.” Cheney, he explained, represented “lots of security and no privacy” while the A.C.L.U. was “lots of privacy but little security.” Post 9/11, Thiel said, it seemed inevitable that the Cheney view would prevail. He then drew another axis, this one with “low-tech” at one end and “high-tech” at the other. “Low-tech” was a catchall for crude, highly intrusive technology. “High-tech,” he said, was more effective but also less invasive. Thiel’s fear was that we would end up with a combination of low-tech and Cheney, in which case civil liberties would likely be crushed. He said that he and Karp wanted to make software that could help save lives but also preserve privacy. “Maybe there were still trade-offs, but they were at a very different level,” he said.

To that end, Palantir’s software was created with two primary security features: Users are able to access only information they are authorized to view, and the software generates an audit trail that, among other things, indicates if someone has tried to obtain material off-limits to them. But the data, which is stored in various cloud services or on clients’ premises, is controlled by the customer, and Palantir says it does not police the use of its products. Nor are the privacy controls foolproof; it is up to the customers to decide who gets to see what and how vigilant they wish to be. The potential for abuse seems vast, especially in the United States, where digital-privacy laws are not nearly as stringent as in Europe. In 2018, Bloomberg Businessweek broke the story of a rogue JP Morgan Chase employee who had used Palantir’s software to spy on colleagues, reading their emails and tracking their movements. Even some of the bank’s senior executives were unknowingly surveilled.

Over the years, Palantir has been embroiled in several controversies that have raised doubts about its own trustworthiness. In 2011, the hacker collective Anonymous released emails it had taken from a third party showing that Palantir employees were involved in a proposed misinformation campaign to discredit WikiLeaks and to smear some of its supporters, notably Glenn Greenwald. Though no one was fired, Karp personally apologized to Greenwald. (When I asked Karp about the episode, he chalked it up to “growing pains.”) Palantir was also implicated in the Cambridge Analytica scandal. Christopher Wylie, the former Cambridge Analytica employee-turned-whistle-blower, claimed that Palantir helped the firm harvest Facebook data that was then used on behalf of the Trump campaign. Palantir, which has a policy of not working on elections, said the matter involved just one employee in its London office and that the person was fired.

For those made nervous by Palantir, the company’s work with police departments has been a source of particular worry. Of all the ways that Big Data can be used, perhaps none generates greater attention than predictive policing, in which quantitative analyses are used to identify places that seem especially prone to crime and individuals who are likely to commit or fall victim to a crime. To critics, data-driven policing encourages overly aggressive tactics and reinforces racial biases that have long plagued the criminal-justice system. Palantir’s effort to market its software to police departments can also be regarded as an example of how weapons originally meant for the war on terrorism are now being deployed on American streets. “This is a tool designed to enhance government surveillance now being redirected on the domestic population,” says Andrew Guthrie Ferguson, a professor of law at American University who has written extensively about policing and technology.

That said, Palantir has struggled to drum up business from police departments. The New York Police Department stopped using Palantir a few years ago, as did the New Orleans Police Department. Not long before, concerns had been raised about how the N.O.P.D. was using the data. These days, the only major metropolitan force using Palantir is the Los Angeles Police Department. Before traveling to Los Angeles last December, I reached out to the L.A.P.D., asking to interview officials about Palantir. My request was turned down. But Sarah Brayne, now an assistant professor of sociology at the University of Texas at Austin, had better luck.

In 2013, when Brayne was a doctoral candidate at Princeton researching the use of data in policing, members of the L.A.P.D. allowed her to study their deployment of new technologies. Over the next two years, she enjoyed considerable access to the department, interviewing dozens of officers and going on ride-alongs in patrol cars. It became apparent to her that Palantir’s software was having a significant impact. For instance, Palantir’s network analysis — its ability to identify a person’s friends, relatives, colleagues and other relations — was pulling people into the L.A.P.D.’s surveillance system who otherwise wouldn’t have been.

Brayne’s findings will be included in a book coming out next month called “Predict and Surveil.” In it, she quotes one L.A.P.D. captain who inadvertently confirmed the worst suspicions about the use of data analytics in policing. “Let’s say I have something going on with the medical-marijuana clinics where they’re getting robbed,” he told Brayne. “I can put in an alert to Palantir that says anything that has to do with medical marijuana plus robbery, plus male, Black, six foot. I like throwing the net out there, you know?” Racial profiling was just one obvious risk. Among the many data streams available to the L.A.P.D. via Palantir were automatic license-plate readers, and it was easy enough to conjure nightmare scenarios. A detective could conceivably use that information to squeeze a reluctant witness — say, by finding out he was having an affair. Someone in the L.A.P.D. could possibly keep tabs on his ex-wife’s comings and goings. Brayne told me that what most troubled her about the L.A.P.D.’s use of data was its opaqueness. “Digital surveillance is invisible,” she said. “How are you supposed to hold an institution accountable when you don’t know what they are doing?”

A great deal of the controversy that dogs Palantir can also be attributed to Thiel, whose activities have raised some doubt about his commitment to democratic society and fair play. In the past, Thiel has argued that democracy and economic freedom are incompatible and suggested that giving women the vote had undermined the latter. After Gawker reported that he was gay, he secretly financed the Hulk Hogan lawsuit that bankrupted the website. Last year, The Wall Street Journal reported that Thiel had been urging Mark Zuckerberg not to censor political ads on Facebook. Thiel’s comments and activities occasionally get in the way of Palantir’s messaging. Karp gave a talk in Washington in September last year in which he said that the only justifiable use of facial-recognition technology by law enforcement was to exonerate people. A few months later, The Times reported that Thiel had helped finance a start-up called Clearview AI, whose facial-recognition app was being used by police departments around the country to charge individuals with crimes. Thiel’s investment in Clearview seemed to contradict Karp’s position and also raised questions about the sincerity of the views he had expressed regarding civil liberties and privacy.

When I asked Thiel about the risk of abuse with Palantir, he answered by referring to the company’s literary roots. “The Palantir device in the Tolkien books was a very ambiguous device in some ways,” he said. “There were a lot of people who looked into it and saw more than they should see, and things went badly wrong when they did.” But that didn’t mean the Palantir itself was flawed. “The Tolkien point I always make is that at the end of the day, it was actually a good device that was critical to the plot of the whole story. The way it worked was that Aragorn looked into the Palantir, and he showed Sauron the sword with which the One Ring had been cut off Sauron’s fingers at the end of the Second Age. This convinced Sauron that Aragorn had the One Ring and caused Sauron to launch a premature attack that emptied out Mordor and enabled the hobbits to sneak in to destroy the One Ring.” He continued: “The plot action was driven by the Palantir being used for good, not for evil. This reflected Tolkien’s cosmology that something that was made by the good elves would ultimately be used for good.”

A moment later, he added: “That’s roughly how I see it, that it is ultimately good and still very dangerous. In some ways, I think that was reflected in the choice of the name.”

In the late aughts, Palantir began pitching its technology to the U.S. military. The Army had equipped its troops with a battlefield-intelligence platform that was doing a poor job of protecting them, but it had sunk billions of dollars into the system and was unreceptive to Palantir. So Palantir started offering its software directly to individual battalions in Iraq and Afghanistan. By the end of 2011, about three dozen units across the military were using Palantir, and some were raving about its ability to steer them clear of ambushes and roadside bombs. According to Fortune magazine, a few senior military figures had become fans, too, among them Gen. James Mattis, Lt. Gen. H.R. McMaster and Lt. Gen. Michael Flynn.

In 2012, the Army commissioned an assessment of Palantir. According to a draft of the report that Palantir produced during litigation, 96 percent of military personnel surveyed deemed Palantir’s software to be effective. But rather than embracing Palantir, officials appeared to ignore the report. Two years later, the Army finally conceded that the intelligence system it was providing to troops was inoperable and began soliciting bids to develop a replacement. It refused, however, to allow Palantir to take part because its software was an off-the-shelf product, and the Army was only willing to entertain proposals for building a new system from scratch. In June 2016, Palantir sued the Army, and three months later, a federal court ruled in its favor. The judge said the Army had acted in “an arbitrary and capricious manner” and ordered it to open up the competition to Palantir.

The protracted battle with the Army is now corporate lore at Palantir, a story that encapsulates how the company sees itself — the scrappy outsider, dedicated to ensuring that good software triumphs over bad. In truth, the saga was slightly more complicated than that. For one thing, Palantir hired lawyers and lobbyists to plead its case and cultivated some prominent allies, such as Senator John McCain. Jonathan Wong, a former Marine and now a policy researcher at the RAND Corporation, says that the Army wasn’t necessarily acting out of malice toward Palantir. It wanted a more comprehensive battlefield-intelligence system than Palantir was offering at the time, one that could be used against “what we are fighting today and what we will be facing tomorrow,” as he puts it. But Wong, whose dissertation at RAND focused in part on the early relationship between Palantir and the Pentagon, says that Palantir’s software was better for the counterinsurgency and counterterrorism challenges that the military was facing at the time.

The federal court’s decision was handed down eight days before Trump was elected president. Depending on how you see it, the timing was either merely coincidental or portentous. With Trump’s victory, Palantir was suddenly among the best-connected companies in Washington. Thiel had been one of Trump’s most prominent supporters, and Mattis, McMaster and Flynn all ended up with senior positions in the new administration. The Trump years have been a bonanza for Palantir. Since Trump took office, it has won military contracts worth billions, including an $800 million contract to build the replacement battlefield-intelligence system. Palantir also has contracts with a number of civilian departments and agencies, among them the I.R.S., the S.E.C. and the C.D.C. The U.S. government now accounts for around half its business.

There appear to have been no allegations of impropriety around the military contracts that Palantir has won under Trump. But the military procurement process has not been immune to the ethical concerns that have swirled around his presidency. Last year, Amazon filed a lawsuit claiming that it had been passed over for a $10 billion defense contract because the Pentagon had yielded to pressure from Trump, who had repeatedly attacked the company’s founder and chief executive, Jeff Bezos, and who had also publicly stated that he did not want Amazon to get the deal.

Karp told me the idea that Palantir had benefited from Thiel’s support of Trump was “completely ludicrous,” and he bemoaned “the unfairness it creates toward us.” Whatever good will Thiel enjoyed with Trump, he said, was offset by his own opposition. “I think they already know my views at the White House,” he said. “It’s true Peter is chairman, [but] I’m running the company, I don’t have close ties with the Trump administration.” Still, he acknowledged that he was worried about “the guilt by association thing” — the possibility that Palantir could be tarnished by its perceived links to Trump. Even so, he refused to back away from the most damaging connection, Palantir’s work with ICE.

Palantir’s client relationships are sometimes born in moments of crisis. That was true with French intelligence, and it was also the case with ICE, which sought Palantir’s assistance after one of its agents was assassinated by a Mexican drug cartel. According to the company, it took Palantir’s engineers 11 hours to merge all of the relevant data, and within two weeks the killers were identified and arrested. ICE subsequently awarded Palantir a contract to help manage the data of Homeland Security Investigations, or H.S.I., the ICE subdivision that handles drug smuggling, human trafficking, financial crimes and cybercrimes. Palantir’s relationship with ICE attracted little notice before Trump’s presidency. But it became deeply controversial as Trump made good on his campaign pledge to curb illegal immigration.

Initially, Palantir tried to deflect criticism by pointing out that its contracts were with H.S.I., not Enforcement and Removal Operations, or E.R.O., the subdivision that was spearheading Trump’s policy. “We do not work for E.R.O,” the company said in a statement to The Times in 2018. That may have been technically true, but it wrongly implied that Palantir was playing no part in the crackdown. In the years before, H.S.I. had supported E.R.O. in a continuing operation to arrest and possibly deport family members of undocumented children who were caught trying to cross the border. And last year, H.S.I. led a raid on food-processing plants in Mississippi in which nearly 700 people were arrested. Interviewed by CNBC at Davos in January, Karp appeared to concede that Palantir’s previous disavowals were no longer operative. “It’s a de minimis part of our work, finding people in our country who are undocumented,” he said.

But Jacinta González of the advocacy group Mijente contends that even that comment was “totally false” and that Palantir’s software has played an integral role. She notes that ICE itself describes Palantir’s software as “mission critical,” underscoring its importance to the government agency. She claims that in recent years, ICE raids on undocumented individuals became much more targeted — agents seem to know exactly whom they are looking for and where to find them, which had not always been the case. González says it was clear to her and her colleagues that ICE had somehow obtained access to a wealth of personal information about those individuals and had also acquired data-analytics capabilities that allowed it to operate with greater precision. With the help of a research firm that examined government documents, Mijente concluded that Palantir’s software was helping to power ICE’s crackdown. “[Palantir] created something tailor-made for ICE to be able to run the kind of raids it wants,” González says. “To say that they are not included in enforcement is kind of laughable.”

Last year, Mijente and other groups held protests outside Palantir’s offices in New York and Palo Alto, as well as outside Karp’s Palo Alto home (Mijente also organized demonstrations in the lead-up to Palantir’s recent public listing). Student organizations at colleges and universities across the country also spoke out against Palantir. For years, the company had sponsored a conference on privacy law held at the University of California, Berkeley. But the organizers dropped Palantir after participants pressured them to cut ties to the firm. There was also dissent within Palantir: More than 200 employees sent a letter to Karp expressing their concern over ICE. Thiel’s political activities weren’t helping. On one of the days that I was with Karp in Paris, Thiel co-hosted a fund-raiser for a former Kansas secretary of state, Kris Kobach, known for his hard-line anti-immigration position.

A few weeks after I saw Karp in Paris, I visited him at a home that he owns in Vermont. When I drove up the dirt road leading to the cabin-style house, a couple of bodyguards greeted me. Karp’s personal assistant then came out and took me into the house, where Karp was waiting at the dining-room table. Suitcases were lined up against a wall. Karp had spent the morning roller skiing. He arrived the night before, and I assumed he was staying for the weekend. But he was going to Boston later that afternoon for a meeting and then heading to Europe. After lunch, we spent a couple of hours hiking. Two of Karp’s bodyguards trailed us by a few feet while two others waited in the parking lot. We talked at length about ICE. He noted that other tech companies had contracts with ICE, yet activists seemed to be directing most of their ire at Palantir, which he took as a backhanded compliment. “People understand we have these powerful platforms and that the platforms actually work,” he said, adding that maybe the protesters were ignoring the other companies because their technology was “not as effectual.”

Karp made clear that he was opposed to Trump’s immigration policies: “There are lots of reasons I don’t support the president; this is actually also one of them.” He told me that he was “personally very OK with changing the demographics of our country” but that a secure border was something that progressives should embrace. “I’ve been a progressive my whole life,” he said. “My family’s progressive, and we were never in favor of open borders.” He said borders “ensure that wages increase. It’s a progressive position.” When the left refuses to seriously address border security and immigration, he said, the right inevitably wins. To the extent that Palantir was helping to preserve public order, it was “empirically keeping the West more center-left.”

But he claimed that if ICE had sought Palantir’s services after Trump took office, he probably would have balked. “I’m not sure I would feel strongly about doing it,” he said. “We probably wouldn’t do the contract. But that’s different than pulling the plug.” Karp said that Palantir couldn’t break with ICE because doing so would mark it as an unreliable partner in the eyes of the military. If Palantir walked away from ICE, he said, it would send a horrible message to soldiers who depend on Palantir’s software. “Why would a war fighter believe you aren’t going to do the same thing to them when they’re in the middle of a battle?” he asked.

He told me that Palantir has rejected some potential clients because it was worried about how they might use its software. It had spurned a lucrative offer from the Saudi government because of Riyadh’s human rights record, he said, and had likewise turned down a major tobacco company. Karp said that he found it hard personally to see Palantir accused of facilitating racism. But he told me that he had been reluctant to talk about his mother because “I don’t want to instrumentalize her” and also because “I don’t think the emotional argument is as persuasive as people think.”

Instead, Karp was trying to initiate a broader debate about Silicon Valley and U.S. national security. He had a convenient cudgel at his disposal: In 2018, Google withdrew from Project Maven, the Pentagon’s artificial-intelligence program, after facing resistance from some employees, who didn’t think the company should be involved in the development of potentially lethal weapons. Karp had elsewhere criticized Google’s decision as “borderline craven” and had sarcastically called the Google employees “super-woke engineers.” In his view, Project Maven was nothing less than the Manhattan Project of the 21st century, and, as with the atomic bomb, the country that gained a military edge with artificial intelligence would “determine the world order tomorrow.” What he didn’t say publicly was that Palantir had replaced Google on parts of the project. (Karp couldn’t confirm that, but I confirmed it, and it has been reported elsewhere.)

Karp insisted that Palantir was more in step with public opinion in the United States than Google and other Silicon Valley giants were. “We’re making Western institutions strong and, in some cases, dominant,” he said. “That’s our narrative. Now, that’s probably not a popular narrative in the Valley. It’s a very popular narrative in the rest of America. What’s Google’s narrative? ‘We destroy the media, we divide the country, we take away your job, we get rich, and by the way, when the country needs you, we’re nowhere to be found.’” He added that if the “Google standard takes hold, the single biggest strategic asset America has, which is our ability to produce software platforms, will be taken out of the hands of our war fighters. And that de facto means our adversaries are in a much stronger position.”

But if Karp’s broadsides against Google were meant to ease public pressure on Palantir, it didn’t seem to work. When it was announced in April that Palantir had been awarded the H.H.S. contracts, a backlash ensued. Progressive groups, human rights organizations and members of Congress criticized the deals. The main concern was that the Trump administration might use information collected by H.H.S. to target immigrants. In a six-page letter sent in July to the H.H.S. secretary, Alex Azar, Senator Elizabeth Warren and 15 other congressional Democrats cited the immigration issue and said that their reservations about the deal were “compounded by the fact that Palantir has a history of contracting with ICE.” (An H.H.S. spokesperson says that the data it collects as part of its Covid response does not include personally identifiable information and is not being shared with ICE.) Last month, Representatives Alexandria Ocasio-Cortez and Jesús García, pointing to Palantir’s contracts with H.H.S. and other aspects of its business, asked the S.E.C. to investigate the company before allowing it to go public. Their request evidently had no effect, but it was an indication of how Palantir has come to be viewed in progressive circles.

Palantir is not without Democratic allies. James Carville is an informal adviser to the company, and Palantir’s technology was used extensively by the Obama administration. When Senator Kamala Harris, the Democratic vice-presidential nominee, was California’s attorney general, her office turned to Palantir for help creating a statewide law-enforcement database. But living down the controversy over ICE may not be easy. Perhaps an interesting analogue, raised in a column last year by the Times writer Kevin Roose, is Dow Chemical, which produced the napalm that the U.S. military used in Vietnam — a fact that would gut its reputation for decades to come. Even if Palantir’s business ultimately doesn’t suffer, its image has unquestionably been harmed.

In June, Palantir filed to go public, and its stock debuted on Sept. 30. The company eschewed an initial public offering in favor of a direct listing, in which no new shares were created or issued. In the prospectus that it submitted to the Securities and Exchange Commission, Palantir announced that it had moved its headquarters from Palo Alto to Denver, formalizing its break from Silicon Valley. Karp used his introductory letter to drive home the point. He slammed what he called “the engineering elite of Silicon Valley,” said that Palantir had found itself increasingly alienated from the values of the tech industry and reaffirmed the company’s commitment to working with the U.S. military and to defending the West. “We have chosen sides,” Karp wrote, a comment that seemed to imply that Silicon Valley had chosen the opposite side.

The decision to go public represented an about-face for a company that had resisted doing so even as it was being hyped as a next-generation Silicon Valley unicorn. Just before I saw Karp in Paris, he had announced to employees that the company would be remaining private for awhile longer. Karp told me that Palantir had never had trouble raising money and that he worried about the effect a public listing might have on its culture. Thiel just didn’t think that the time was right. He said that “it still feels to me like we’re nowhere near the point where this is the kind of ubiquitous platform” that would, in his view, have justified taking Palantir public.

A few days after Palantir’s stock debuted, I spoke to Karp via video link. He was in Switzerland. He had spent the first few months of the pandemic at a house he owns in New Hampshire, not far from his place in Vermont, before leaving for Europe in July. He was in good spirits, as you would expect of someone who had officially just become a billionaire (he owns at least 6 percent of Palantir). He said that, as an introvert, the lockdown had been no great burden for him. He missed, however, seeing his parents (both reside in the Philadelphia area) and his colleagues.

Some observers had suggested that Palantir’s decision to go public had been driven in part by the prospect of a Trump loss; it was cashing in while its government business was still flourishing. But Karp insisted the election had nothing to do with it — federal contracts, he said, were largely apolitical, and a change in the White House was unlikely to affect Palantir. He also noted that he was supporting Biden and was about to make a donation to his campaign. He told me that Palantir had gone public because its business had matured to the point that it now made sense. Despite the pandemic, he said, the company’s revenue had been up 49 percent during the first six months of the year. More important, he added, Palantir had tweaked its Foundry software so that it could now be installed and updated remotely, which would make it easier to win new business. “Out of nowhere,” he said, “the company was in a technical and financial position” to go public.

Investors weren’t so sure. In its prospectus, Palantir reported that it was still losing hundreds of millions of dollars: $580 million in 2019, following a similar loss the year before. It also disclosed that just three customers accounted for roughly 30 percent of its revenue. (Palantir’s governance structure was another concern: Karp, Thiel and Stephen Cohen will retain just under 50 percent of the voting power, as long as their holdings of securities meet a certain minimum.) For years, Palantir had often been portrayed as a colossus. But its financials appeared to tell a different story, and some critics went so far as to suggest that Palantir was barely a viable business. The day before the company’s stock started trading, CNBC interviewed Scott Galloway, a professor at New York University’s Stern School of Business, who was caustic. Palantir, he said, was “crap being flung at tourists to the unicorn zoo.”

To Galloway, Palantir is just more Silicon Valley smoke and mirrors (even if it is no longer based in the valley). To Karp, Palantir is nothing less than a bulwark of liberal democracy — and maybe all that stands between him and the realization of his worst fear, a world succumbing to fascism. To Ben Wizner, the director of the American Civil Liberties Union’s Speech, Privacy and Technology project, Palantir’s business model is predicated on an assumption that its clients should have “legitimate access to every detail of our lives,” and the company’s software is a mechanism by which the government can keep an ever closer watch on us. To Airbus, Palantir is a tool that enhances efficiency and profitability. To Mijente, Palantir is an accessory to human rights violations. To the United Nations World Food Program, which earlier this month was awarded the Nobel Peace Prize for its work on Covid-19 relief, Palantir’s technology has played a key role in its effort to get food and supplies distributed amid the pandemic.

What Palantir does is complicated and mysterious. As with the magical stones for which it is named, people seem to see in it what they want to see. I thought Karp put it pretty nicely. “Palantir,” he said, “is the convergence of software and difficult positions.”

Michael Steinberger is a regular contributor to the magazine. His last feature was about the resiliency of the stock market despite the global pandemic.

Type animation by Nikita Iziev.