Select Page
To Build a Better AI Supercomputer, Let There Be Light

To Build a Better AI Supercomputer, Let There Be Light

GlobalFoundries, a company that makes chips for others, including AMD and General Motors, previously announced a partnership with Lightmatter. Harris says his company is “working with the largest semiconductor companies in the world as well as the hyperscalers,” referring to the largest cloud companies like Microsoft, Amazon, and Google.

If Lightmatter or another company can reinvent the wiring of giant AI projects, a key bottleneck in the development of smarter algorithms might fall away. The use of more computation was fundamental to the advances that led to ChatGPT, and many AI researchers see the further scaling-up of hardware as being crucial to future advances in the field—and to hopes of ever reaching the vaguely-specified goal of artificial general intelligence, or AGI, meaning programs that can match or exceed biological intelligence in every way.

Linking a million chips together with light might allow for algorithms several generations beyond today’s cutting edge, says Lightmatter’s CEO Nick Harris. “Passage is going to enable AGI algorithms,” he confidently suggests.

The large data centers that are needed to train giant AI algorithms typically consist of racks filled with tens of thousands of computers running specialized silicon chips and a spaghetti of mostly electrical connections between them. Maintaining training runs for AI across so many systems—all connected by wires and switches—is a huge engineering undertaking. Converting between electronic and optical signals also places fundamental limits on chips’ abilities to run computations as one.

Lightmatter’s approach is designed to simplify the tricky traffic inside AI data centers. “Normally you have a bunch of GPUs, and then a layer of switches, and a layer of switches, and a layer of switches, and you have to traverse that tree” to communicate between two GPUs, Harris says. In a data center connected by Passage, Harris says, every GPU would have a high-speed connection to every other chip.

Lightmatter’s work on Passage is an example of how AI’s recent flourishing has inspired companies large and small to try to reinvent key hardware behind advances like OpenAI’s ChatGPT. Nvidia, the leading supplier of GPUs for AI projects, held its annual conference last month, where CEO Jensen Huang unveiled the company’s latest chip for training AI: a GPU called Blackwell. Nvidia will sell the GPU in a “superchip” consisting of two Blackwell GPUs and a conventional CPU processor, all connected using the company’s new high-speed communications technology called NVLink-C2C.

The chip industry is famous for finding ways to wring more computing power from chips without making them larger, but Nvidia chose to buck that trend. The Blackwell GPUs inside the company’s superchip are twice as powerful as their predecessors but are made by bolting two chips together, meaning they consume much more power. That trade-off, in addition to Nvidia’s efforts to glue its chips together with high-speed links, suggests that upgrades to other key components for AI supercomputers, like that proposed by Lightmatter, could become more important.

FTX Founder Sam Bankman-Fried Sentenced to 25 Years in Prison

FTX Founder Sam Bankman-Fried Sentenced to 25 Years in Prison

The defense’s court filing was supplemented with letters from Bankman-Fried’s family members and various associates, testifying to his good character, remorse, and utilitarian ideals. “The public perception of Sam could not be further from the truth,” wrote Barbara Fried, his mother. “Being consigned to prison for decades will destroy Sam as surely as would hanging him, because it will take away everything in the world that gives his life meaning.”

Bankman-Fried’s counsel argued for a shorter sentence given that FTX creditors are on track to recover their money in full at the end of the bankruptcy process—although not everyone’s happy with the outdated valuation of the assets, given crypto’s recent meteoric rise in price. They also dismissed the government’s claim that Bankman-Fried would reoffend if allowed to reenter society too quickly as “conjecture on top of hypothetical on top of supposition.” Bankman-Fried deserved no more than six-and-a-half years in prison, his counsel claimed.

The judge was unsympathetic. “There is a risk this man will do something very bad in the future,” Kaplan said, homing in on Bankman-Fried’s appetite for risk. He described Bankman-Fried as a “math nerd” whose decision-making framework was guided primarily by “EV,” or expected value. “In other words, this is a man willing to flip a coin as to the chance of life’s continued existence on Earth. That’s a leitmotif of this entire case,” he said.

The problem for Bankman-Fried is that he can “never put the toothpaste back in the tube,” says Paul Tuchmann, another former US prosecutor and partner at law firm Wiggin and Dana. The fact that FTX users are set to recover money at an unspecified future date “does not nearly undo the harm they suffered” in the intervening period, he says. In one victim impact statement, an FTX customer said they had subsisted on ham, cheese, and ketchup sandwiches after the exchange’s collapse. In another, John Ray III, the restructuring professional steering FTX through bankruptcy, wrote that customers “will never be returned to the same economic position they would have been in today absent [Bankman-Fried’s] colossal fraud,” because the bankruptcy claims aren’t based on current crypto values.

The US Department of Justice made play of these issues in its own presentence filings, pressing home the gravity of Bankman-Fried’s crimes, the range and number of his victims, and the way he obstructed the investigation by allegedly giving “false testimony” on the stand.

The government also underlined the need to deter would-be crypto fraudsters, suggesting that “some individuals have operated under the misimpression that they are unregulated, not subject to criminal laws, or can avoid scrutiny or significant jail time.” Until the fall of FTX, the DOJ had secured few landmark crypto convictions, despite forming a specialist crypto-crime task force in 2021. But in sentencing Bankman-Fried, who had become an almost messianic figure in crypto, the judge could elect to “send a message” to the industry, says Tuchmann.

With sentencing complete, Bankman-Fried will be returned to the temporary holding facility in which he has been kept since his arrest, until the Federal Bureau of Prisons selects a permanent destination. The judge recommended Bankman-Fried be housed in a low-to-medium-security facility as close as possible to the San Francisco Bay Area, where his parents reside. A decision will be reached within the next few months.

In the federal system, there is no possibility of parole. The best Bankman-Fried can hope for—short of winning on appeal—is early release for good behavior.

The DOJ has frequently compared the FTX founder to Ponzi fraudster Bernie Madoff, who received a prison sentence of 150 years. But even the sentence requested by Bankman-Fried’s counsel feels long, says Naftalis, given the differences between the two cases. “This isn’t Madoff,” he says. “SBF was on top of crypto, but crypto is not Wall Street. Let’s remember that.”

In whatever facility, Bankman-Fried’s incarceration will be far from comfortable. “Just think about it,” says Naftalis. “A day in jail is a long time.”

This is a developing story. Please check back for updates.

The NSA Warns That US Adversaries Free to Mine Private Data May Have an AI Edge

The NSA Warns That US Adversaries Free to Mine Private Data May Have an AI Edge

Electrical engineer Gilbert Herrera was appointed research director of the US National Security Agency in late 2021, just as an AI revolution was brewing inside the US tech industry.

The NSA, sometimes jokingly said to stand for No Such Agency, has long hired top math and computer science talent. Its technical leaders have been early and avid users of advanced computing and AI. And yet when Herrera spoke with me by phone about the implications of the latest AI boom from NSA headquarters in Fort Meade, Maryland, it seemed that, like many others, the agency has been stunned by the recent success of the large language models behind ChatGPT and other hit AI products. The conversation has been lightly edited for clarity and length.

Person in a suit smiling in front of the American and National Security Agency flags

Gilbert HerreraCourtesy of National Security Agency

How big of a surprise was the ChatGPT moment to the NSA?

Oh, I thought your first question was going to be “what did the NSA learn from the Ark of the Covenant?” That’s been a recurring one since about 1939. I’d love to tell you, but I can’t.

What I think everybody learned from the ChatGPT moment is that if you throw enough data and enough computing resources at AI, these emergent properties appear.

The NSA really views artificial intelligence as at the frontier of a long history of using automation to perform our missions with computing. AI has long been viewed as ways that we could operate smarter and faster and at scale. And so we’ve been involved in research leading to this moment for well over 20 years.

Large language models have been around long before generative pretrained (GPT) models. But this “ChatGPT moment”—once you could ask it to write a joke, or once you can engage in a conversation—that really differentiates it from other work that we and others have done.

The NSA and its counterparts among US allies have occasionally developed important technologies before anyone else but kept it a secret, like public key cryptography in the 1970s. Did the same thing perhaps happen with large language models?

At the NSA we couldn’t have created these big transformer models, because we could not use the data. We cannot use US citizen’s data. Another thing is the budget. I listened to a podcast where someone shared a Microsoft earnings call, and they said they were spending $10 billion a quarter on platform costs. [The total US intelligence budget in 2023 was $100 billion.]

It really has to be people that have enough money for capital investment that is tens of billions and [who] have access to the kind of data that can produce these emergent properties. And so it really is the hyperscalers [largest cloud companies] and potentially governments that don’t care about personal privacy, don’t have to follow personal privacy laws, and don’t have an issue with stealing data. And I’ll leave it to your imagination as to who that may be.

Doesn’t that put the NSA—and the United States—at a disadvantage in intelligence gathering and processing?

II’ll push back a little bit: It doesn’t put us at a big disadvantage. We kind of need to work around it, and I’ll come to that.

It’s not a huge disadvantage for our responsibility, which is dealing with nation-state targets. If you look at other applications, it may make it more difficult for some of our colleagues that deal with domestic intelligence. But the intelligence community is going to need to find a path to using commercial language models and respecting privacy and personal liberties. [The NSA is prohibited from collecting domestic intelligence, although multiple whistleblowers have warned that it does scoop up US data.]

Google Used a Black, Deaf Worker to Tout Its Diversity. Now She’s Suing for Discrimination

Google Used a Black, Deaf Worker to Tout Its Diversity. Now She’s Suing for Discrimination

Hall says when she has access to an interpreter, they are rotated throughout the week, forcing her to repeatedly explain some technical concepts. “Google is going the cheap route,” Hall claims, saying her interpreters in university were more literate in tech jargon.

Kathy Kaufman, director of coordinating services at DSPA, says it pays above market rates, dedicates a small pool to each company so the vocabulary becomes familiar, hires tech specialists, and trains those who are not. Kaufman also declined to confirm that Google is a client or comment on its policies.

Google’s Hawkins says that the company is trying to make improvements. Google’s accommodations team is currently seeking employees to join a new working group to smooth over policies and procedures related to disabilities.

Beside Hall’s concerns, Deaf workers over the past two years have complained about Google’s plans—shelved, for now—to switch away from DSPA without providing assurances that a new interpreter provider would be better, according to a former Google employee, speaking on the condition of anonymity to protect their job prospects. Blind employees have had the human guides they rely on excluded from internal systems due to confidentiality concerns in recent years, and they have long complained that key internal tools, like a widely used assignment tracker, are incompatible with screen readers, according to a second former employee.

Advocates for disabled workers try to hold out hope but are discouraged. “The premise that everyone deserves a shot at every role rests on the company doing whatever it takes to provide accommodations,” says Stephanie Parker, a former senior strategist at YouTube who helped Hall navigate the Google bureaucracy. “From my experience with Google, there is a pretty glaring lack of commitment to accessibility.”

Not Recorded

Hall has been left to watch as colleagues hired alongside her as content moderators got promoted. More than three years after joining Google, she remains a level 2 employee on its internal ranking, defined as someone who receives significant oversight from a manager, making her ineligible for Google peer support and retention programs. Internal data shows that most L2 employees reach L3 within three years.

Last August, Hall started her own community, the Black Googler Network Deaf Alliance, teaching its members sign language and sharing videos and articles about the Black Deaf community. “This is still a hearing world, and the Deaf and hearing have to come together,” she says.

On the responsible AI team, Hall has been compiling research that would help people at Google working on AI services such as virtual assistants understand how to make them accessible to the Black Deaf community. She personally recruited 20 Black Deaf users to discuss their views on the future of technology for about 90 minutes in exchange for up to $100 each; Google, which reported nearly $74 billion in profit last year, would only pay for 13. The project was further derailed by an unexpected flaw in Google Meet, the company’s video chat service.

Hall’s first interview was with someone who is Deaf and Blind. The 90-minute call, which included two interpreters to help her and the subject converse, went well. But when Hall pulled up the recording to begin putting together her report, it was almost entirely blank. Only when Hall’s interpreter spoke did the video include any visuals. The signing between everyone on the call was missing, preventing her from fully transcribing the interview. It turned out that Google Meet doesn’t record video of people who aren’t vocalizing, even when their microphones are unmuted.

6 Months After New York Banned Airbnb, New Jersey Is Doing Great

6 Months After New York Banned Airbnb, New Jersey Is Doing Great

More than 95 percent of the group’s members say they have no intention of becoming long-term landlords, says Lindsay. Instead, he argues, they are now faced with rising housing costs and no immediate way to offset them. The law “has yielded some unintentional effects that are harming smaller homeowners,” Lindsay says.

Amid the uncertainties, there may be some winners from the law: hotels in the city and the state of New Jersey. Hotel occupancy rates in New York have been slightly up year-over year, by 4 percent in January and 3.4 percent through February 24, according to CoStar, which tracks commercial real estate. The average daily room rate in January was up from $198 a night to $209, and from $200 to about $207 through February 24.

Across the Hudson River, demand for short-term rentals has risen sharply in Jersey City, Hoboken, and Weehawken since the law passed, all cities that offer quick access into downtown Manhattan. Jersey City has seen demand rise 77 percent year-over-year as of mid-February, according to AirDNA, while in Weehawken and Hoboken demand has increased 45 and 32 percent, respectively.

The high rents in New York so far seem unaffected. Despite hopes from lawmakers that the ban might bring them down, short-term rentals are just one piece of a complex unaffordable housing problem. More than half of New York households are rent-burdened, meaning they spend more than 30 percent of their income on housing, a 2023 report from nonprofit Community Service Society found.

The median rent of properties in the city on Zillow was up $165 in March from the same month last year, coming to $3,465. But a January 2024 report from real estate company Douglass Elliman found that rent prices fell in Manhattan and Brooklyn, areas popular with tourists, after rents stabilized and the number of vacant apartments increased in December. If restricting short-term rentals helps residents, it may take longer than six months to manifest. A recent study looked at Irvine, California, which bans short-term rentals in all residential zones, and found that after two years of the ban, rents dropped by about 3 percent.

Enforcement of the law has been patchy. With Airbnb off limits, people turned to Craigslist, Facebook Marketplace, or other home-sharing sites like Houfy to list their apartments after they were booted from sites like Airbnb or Vrbo. The city has not yet issued any fines to people for renting out their apartments illegally, as it is still working on compliance, according to Christian Klossner, executive director of the Mayor’s Office of Special Enforcement, which oversees the licensing process. But he says the city is responding to complaints related to illegal renting. As of February 26, the city had received 5,783 applications to run short-term rentals. It has approved 1,594, denied 990, and sent back more than 3,000 for more information or corrections.

Airbnb opposed the law, and sued the city before it took effect, but the case was dismissed last August. Now that the law is in effect, the company is maintaining its opposition. “In the six months since New York City’s short-term rental rules went into effect, we’ve seen travelers facing record hotel prices and former hosts struggling with loss of income—but we have seen no improvement in housing costs,” Nathan Rotman, Airbnb’s Northeast policy lead, tells WIRED. “We hope city leaders listen to hosts who are advocating for changes to the existing rules.”

Lindsay, of the homeowners association, says people like him are hurting while their counterparts in New Jersey benefit. Renting out an apartment on Airbnb “was a lifeline for me, especially during the pandemic,” he says. The association is working on ways the New York City Council might amend the law to allow these smaller hosts to operate short-term rentals. Right now, he says, it fails by grouping small homeowners in with big-time investors. “It treats all property owners as if they’re these evil, maniacal villains.”