In a captivating Friday evening late in August, Elon Musk embarked on a journey that would redefine the future of autonomous driving. Seated comfortably in his Model S at Tesla’s Palo Alto headquarters, Musk embarked on a journey that was nothing short of extraordinary, entrusting his car to the marvel of Full Self-Driving (FSD) technology. As Mozart’s soothing melodies filled the cabin, he livestreamed this remarkable voyage, with one intriguing highlight: a casual pass by the residence of Mark Zuckerberg, punctuated by playful banter about a cage-match showdown. “Perhaps I should knock on the door and propose a hand-to-hand combat challenge,” Musk mused, his laughter echoing before he resumed entrusting the car’s autonomous capabilities.
What made this journey truly groundbreaking was FSD 12, a radical leap forward in autonomous technology. Unlike its predecessors, this new version was not crafted from hundreds of thousands of lines of code. Instead, it had learned to drive by processing billions of frames of human-like behavior, akin to how advanced language models like ChatGPT learn from vast volumes of text.
Astoundingly, this transformative shift had occurred just eight months earlier.
Dhaval Shroff, a young member of Tesla’s autopilot team, likened this innovation to “ChatGPT, but for cars.” Instead of relying on rule-based algorithms, Tesla’s new “neural network planner” followed a path of human imitation. Shroff explained, “We process an enormous amount of data on how real human drivers acted in complex driving situations, and then we train a computer’s neural network to mimic that.” It was a departure from conventional wisdom, mirroring how humans learn from one another, as envisaged by Alan Turing in his 1950 paper, “Computing Machinery and Intelligence.”
By early 2023, the neural network planner project had analyzed a staggering 10 million video clips from Tesla customers’ cars. However, it wasn’t merely about mimicking average human drivers; the bar was set higher. Musk’s directive was clear: seek out actions that a “five-star Uber driver” would take. This meticulous curation ensured that only the finest examples of human driving prowess shaped the system’s learning.
As Musk delved deeper into this new paradigm, he questioned its necessity. Could it be an overly complex solution? Musk, known for his maxim of not using a cruise missile to kill a fly, sought clarity. Shroff’s response was compelling. He demonstrated scenarios where the neural network planner outperformed rule-based systems, such as navigating through a chaotic road strewn with obstacles. It was the difference between precision and chaos. Musk, ever the visionary, even envisioned a James Bond-style demonstration, complete with explosions and a UFO, all while the car navigated unscathed.
To gauge progress, Musk established a clear metric: the number of miles FSD cars could travel without human intervention. He likened it to a video game, where the score was visible daily, injecting motivation into the team. Enormous 85-inch monitors adorned the workspace, displaying real-time data on the cars’ autonomous performance. A celebratory gong awaited, to be struck whenever a problem was solved.
By mid-April 2023, it was Elon Musk’s turn to experience the new neural network planner firsthand. Accompanied by Tesla’s director of Autopilot software, Ashok Elluswamy, and members of the Autopilot team, Musk embarked on a journey that showcased the system’s prowess. The car’s autonomous navigation, handling complex maneuvers and unexpected obstacles, left Musk in awe. His hands never touched the wheel, and he was amazed when the car made a decision he thought was better than his own.
“Amazing work, guys,” Musk exclaimed at the end of the ride, as they headed to the weekly Autopilot team meeting. The room buzzed with anticipation. Musk, now a believer, urged them to push the neural network project forward.
During the discussion, a pivotal fact emerged: the neural network required training on at least a million video clips to excel. Tesla’s unique advantage was its fleet of nearly 2 million vehicles worldwide, continually collecting video clips. “We are uniquely positioned to do this,” Elluswamy affirmed.
Four months later, the neural network had matured to replace the old approach and become the foundation of FSD 12, poised for release upon regulatory approval. Yet, one challenge remains: human drivers’ idiosyncrasies, such as rolling through stop signs. The National Highway Safety Board is contemplating whether such human quirks should extend to self-driving cars.
In the quest for the autonomous future, Tesla’s FSD 12 stands at the forefront, a testament to innovation, human emulation, and the relentless pursuit of perfection. As the world watches, Tesla drives us closer to a future where cars drive themselves with the grace and wisdom of the finest human drivers.