Ethernet’s 40th Anniversary: A Conversation Between Bob Metcalfe and Jensen Huang
Date: October 2020
Occasion: 40th Anniversary of the First Ethernet Specification (September 30, 1980)
Introduction
Jensen Huang: Well Bob, it is such an incredible pleasure to be able to spend time with you today on the 40th anniversary of the first Ethernet specification. You’re the father of Ethernet, you’re the inventor of Ethernet. Of course, as always, we have lots and lots of collaborators, but your contribution to the foundation of the internet today—its impact can’t possibly be overstated.
You’ve had an amazing career. You were born in 1946, which is also the birth year of the transistor. The two of you enabled the internet that we know today. You went on to an incredibly celebrated career—you’re a recipient of countless awards: the National Medal of Technology, the IEEE Medal of Honor, the Internet Hall of Fame, the Alexander Graham Bell Medal, the Grace Murray Hopper Award, and countless computer science awards that can’t possibly fully recognize the contribution you’ve made to the world and to society today.
You know very well that the invention of Ethernet is what enabled the internet today, where billions of people are connected to it, trillions of dollars of the economy is enabled by it. Because of the internet, it’s possible for us to be in the same digital space, although not in the same physical space, and to enjoy each other’s presence. Because of the internet, it’s possible for companies to become global companies, virtual giant companies with sites in Silicon Valley, China, India, Europe—sites all over the world to form today’s great companies.
When you go back and think about the beginning of it, what inspired you? What was the inspiration behind the invention of Ethernet? What was the problem you were trying to solve, and what were the surrounding conditions at Xerox PARC that inspired you to invent Ethernet?
Early Days: From ARPANET to Ethernet
Bob Metcalfe: Thank you for those many kind words, Jensen. I’m honored to be with you here today.
I got into the networking business in 1969 because I showed up at Harvard as a grad student. The algorithm that grad students follow is they find out where the money is and then they decide to major in that. The Advanced Research Projects Agency of the Department of Defense was giving out money to computer science departments. I detected that, so I became an ARPANET grad student. At MIT, as a student at Harvard but as an employee of MIT, I connected MIT to the ARPANET, the then-internet. I learned in the course of that how to send bits one at a time down a very long wire.
Then I moved to Xerox Corporation Palo Alto—Xerox PARC—where I put PARC on the ARPANET. I just did the same thing, built the same hardware.
Jensen: When you put PARC on the ARPANET, was it one computer? What was it that you put on the ARPANET?
Bob: It was one computer. Her name was MAXC. It was a PDP-10 clone that we built in that lab. I was the networking guy in the Computer Science Lab there, so I got the job of connecting it to the ARPANET—basically building a printed circuit card about this big that would connect the time-sharing system, the PDP-10 running the TENEX time-sharing system, connecting it to the ARPANET packet switch in the same room. So once again, sending bits one at a time down a long wire.
Jensen: So you were a system designer as well as the network administrator, so to speak?
Bob: I operated at the boundary of hardware and software. I would build hardware, I would write software, at the very least the plumbing level. My whole career has been at the plumbing level.
Jensen: Well, you’re the world’s most famous plumber—there’s no doubt. You and Mario both!
Now this printed circuit board that you made was made out of what kind of technology at the time? TTL, MSI-type technology?
Bob: Well, actually I just lied. I didn’t make a printed circuit card—I made a wire-wrapped, big wire-wrapped card using the Texas Instruments 7400 series MSI chips.
Jensen: Oh fantastic! I remember that set. What was it—the 7404 had two flip-flops on it, and the 7400 had six inverters?
Bob: Getting the chips on the card was part of the limiting factor.
The Birth of Ethernet
Bob: Let me move on to Ethernet. After we put Xerox on the ARPANET, we then decided to put a personal computer on every desk. I lucked out—they gave me the job of connecting them together.
Jensen: Bob, at this time the PC hasn’t been invented yet. You were at PARC. The personal computer—was it called a personal computer? It was probably the Alto that you’re talking about?
Bob: It was the Alto. We did call it a personal computer. By the way, you could get into a big argument about which was the first personal computer, but the Alto at Xerox PARC is my version of the first personal computer. And I lucked out—I got to build a network.
One of the requirements was that we were also building the world’s first laser printer: a page per second, 500 dots per inch. If you do the math—500 times 500 times 8.5 times 11 per second—it’s about 20 megabits per second. So the network had to be fast. We couldn’t use RS-232, which was the then-standard.
In my office the day before Ethernet, I had a 300-baud dumb terminal. When we installed Ethernet in my office, I had a 2.94 megabit per second packet-switched LAN in my office. To save you the arithmetic, the increase was a factor of 10,000!
Jensen: Well, it took us another 40 years to do much better than that—but we did! There’s a group doing 800 gigabits per second Ethernet right now.
Bob: That’s 100,000 times, about. 47,000 times, but who’s counting?
Jensen: My first experience with Ethernet was as a local area network—a LAN—really about connecting computers that were close by. My first use of it was connecting peripherals.
From that point in time to where we are today, Ethernet is clearly not just about peripherals, not just about local area networks, not even about coax and twisted pairs—it’s so much more than that. The first step was the standardization of the technology you invented. What was happening at the time? What inspired you to go to the IEEE and make it eventually 802.3?
Standardization: The DEC-Intel-Xerox Alliance
Bob: I left Xerox in January of 1979 with the intent of starting a company, although I had no clue what it would be. In a meeting in February of that year with Gordon Bell, who was then the vice president of R&D of Digital Equipment Corporation, the number two computer company—
Jensen: I know Gordon very well.
Bob: Gordon asked me to design an Ethernet for DEC, and I said I couldn’t do that, Gordon. First of all, I feel loyal to Xerox, and second of all, I’ve already designed the best network I know how, and yours would be second best.
So together we had the idea of writing a letter to Xerox proposing that DEC and Xerox would work together to get their products to be compatible over Ethernet. Then I ran into Intel Corporation down at the National Bureau of Standards, looking for a new chip standard to implement with some new NMOS thing they had.
Pretty soon we had DEC, Intel, and Xerox cooperating on connecting with Ethernet. Suddenly they had a problem—they couldn’t meet because in order to meet, they would have to violate antitrust law. I got my lawyer, a fraternity brother I might add, Howie Charney, and he gave me advice on what I should tell the lawyers at those three companies that would make it okay for them to meet.
I gave them that list of five: no marketing people allowed in the meeting, no pricing discussions, and the goal had to be an industry standard. That was the moment in which the idea of creating an industry standard Ethernet came up.
You have to remember, in those days standards were not made like they are today. IBM would just announce its next product and that would be a standard right away. The idea of bringing Ethernet to the IEEE was an important breakthrough idea.
Jensen: Bob, at this time in your life, how old were you?
Bob: I was thirty-three.
Jensen: Incredible. At that time there weren’t that many technology startups, and surely at 33 that’s quite young. Who did you work with at Intel? For gossip’s sake, who was CEO at the time?
Bob: Andy Grove was the principal mover, but a guy named Kaufman—I’ve forgotten his first name—he was Grove’s right-hand man. He came to a conference pitch I gave on Ethernet, then he invited me back to Intel to give a presentation. I met Mr. Grove, and they were instrumental in getting Ethernet standardized.
Competing Standards: Token Ring and Token Bus
Jensen: At the time there were a couple of other competing standards being proposed—Token Ring and Token Bus. How did the community decide that the standard would become Ethernet? Was it their technical merits, economic merits, industrial support merits?
Bob: On September 30th, 1980—the event we’re celebrating today—DEC, Intel, and Xerox produced a spec, the Ethernet Blue Book it was called, and gave it to the newly forming committee IEEE 802, with no dots, just 802. We thought that Project 802 would then standardize officially Ethernet.
But IBM and General Motors had different ideas. They didn’t like the idea of somebody else making standards for them, so they each submitted a competing proposal: IBM’s Token Ring and General Motors’ Token Bus.
The IEEE then began to consider all three of them and eventually—I didn’t like this outcome—but eventually standardized all three. That’s where the dots come from: 802.3 was Ethernet, 802.4 was the Token Bus, and 802.5 was the IBM Token Ring. The way we proceeded from there was fierce competition for the next 20 years.
Jensen: If I were a startup at that time and it went through the standards bodies and IBM’s was 802.5, I might be inspired to go build that, frankly. What caused Ethernet to gain so much industry support? I know that one of the things you always believed in was interoperability and compatibility. For any platform or standard, that’s the essential core value that allows an industry to be formed around it. But at that point, IBM put out Token Ring to be standardized—why wouldn’t anybody want to build peripherals or computers that would be connected to the world’s leading computing company at that time?
Bob: IBM wasn’t used to fighting standards efforts like Ethernet, and their heart really wasn’t in it. Their dark black heart was intent on pretending to have a standard but shipping a product that was not standard. The way I know this is that through 3Com, my company, we shipped IBM Token Ring ahead of IBM, and we had a hell of a time selling it. We never sold very much of it because IBM, whose heart was not in interoperability, had software dust that they sprinkled over their Token Rings. Whenever we tried to sell a Token Ring into a customer, we couldn’t get it to work because they had this software dust they called SNA—System Network Architecture. Just to annoy them, I used to call it SNOT—SNA over Token.
Jensen: You hadn’t thought of that after all these years!
Bob: Going back to your question, the reason that people went to Ethernet eventually is that, number one, Ethernet was designed in the context of the seven-layer reference model of the internet, and so we knew our place. Ethernet was designed to be level one and level two and not the other levels. IBM didn’t understand this, nor did General Motors.
For example, Ethernet does not have acknowledgments. You send packets—the packets may or may not have acknowledgments, but Ethernet doesn’t know. They’re just packets. The whole idea of acknowledging things was assumed to happen at a higher level, with TCP basically. Whereas IBM, because they were a bit old-fashioned, felt obligated to have acknowledgments. That token in their ring would go around acknowledging that the packet you sent has gotten there, and that was built into level one and two. That was a mistake. IBM Token Ring was always slower and more expensive than Ethernet.
Jensen: The wisdom of specifying Ethernet was to go as far up as possible—to specify as much as necessary but not more than that—so that you can enable innovation and evolution on top.
Bob: That was really great wisdom.
Building 3Com and Finding the Market
Jensen: Now 3Com was founded, Ethernet was standardized, you’re competing against IBM. Who were 3Com’s first customers? Who were the people that got you off the ground?
Bob: There were a lot of people over a long time that got us off the ground. I think the Ethernet bandwagon was joined principally by Hewlett-Packard, DEC, Intel, Xerox, and 18 or 19 other companies. So it was all of us versus IBM and a few of its close friends.
The hard part was that there weren’t any PCs, and the whole purpose of Ethernet was to network PCs. There weren’t any! I mean, there were maybe a hundred thousand Apple IIs floating around, but the Apple II was an 8-bit 6502 microprocessor and it wasn’t worthy of a 10 megabit per second Ethernet.
So we had to wait for the IBM PC to come out, which was announced in August of ‘81. Then we built a card that we shipped in September of ‘82 that you could plug into an IBM personal computer and put it on the Ethernet.
Then it got difficult because people would say, “Well, what good is Ethernet?” And I would say, “It’s good for everything! It’s great! It’s a horizontal platform!” It turns out that is not a good value proposition. If anybody’s planning to say they’re selling a platform that’s good for anything, customers don’t respond to that.
Jensen: They want to know what it’s going to do for them.
Bob: Exactly. Would you like to know how we sold the first Ethernets?
Metcalfe’s Law: The Birth of Network Economics
Jensen: This is such a great story, Bob. This is really your second invention. The first invention was the technology, and your second invention—arguably even more insightful—was understanding what Ethernet really is. Because Ethernet could be many things: it’s a technology, it’s a standard, it’s a networking market, it’s a platform. I think it’s all of that.
Your insights—and I’m going to let you tell it—but your insights about network value, and you were at the time in sales and marketing, you were probably the most educated sales and marketing person in Silicon Valley at the time. You were trying to evangelize a brand new way of thinking about computing. That intersection between your background and your ability to think about customers and creating markets came together at that time.
Tell us the story of how it came about—the value of a network.
Bob: We were selling Ethernet to people with personal computers, of whom there weren’t any. So we had to make it easy to try our product. We made a trial kit—a three-node kit with three cards, option cards, and a diskette with software on it. This cost $3,000. What you could do with it was share a printer among the three PCs, share a disk among the three PCs, send emails among the three PCs, etc.
We sold a bunch of these. It was only $3,000, so people were curious and bought it. Then they all came back and said, “It works just like you said it would. We can share the printers, we can share the disk, blah blah blah. And by the way, sending email among three people isn’t that interesting. Your product isn’t useful to us.”
I was then, as you pointed out, the head of sales and marketing, so that was my problem. One night I went over to Stanford—Xerox had given Stanford some Altos—so I snuck in and used the Alto, which I knew how to use, and I made a slide. A 35-millimeter slide, not a PowerPoint—a 35-millimeter slide.
The slide showed that the cost of a network, about a thousand dollars a card, was linear with the number of nodes you had in your network. But the number of possible connections—which I claimed was the value of the network—grew as the square, because every node you added could talk to the preceding n minus 1 nodes. So n times n minus 1 is roughly n-squared connections.
I argued that the value of a network grew as the square and therefore would eventually overtake the linear cost at some critical point. By the way, I made six copies of this slide because that’s how big the sales force was!
Jensen: [Laughs]
Bob: The slide basically said, “Your network is not useful because it’s not big enough, and the remedy for that is to buy more of our products.” And they believed us!
Jensen: Who was the first customer that believed you? What was your first large installation?
Bob: I remember a young man in Maryland who worked for a bank. He was going to build a small workgroup of PCs connected by Ethernet with a multi-user accounting package. He may have been an early customer.
By the way, I learned a lesson from him. I learned why people buy products from startups. Why would you take the risk of buying a product from a startup? He explained it to me. He said, “I want to be the president of this bank, and this workgroup that we’re building here—this workgroup multi-user PC network—is going to save us a lot of money and make me look good. I’m going to get promoted and move up.”
The reason people buy cutting-edge technology is because they have competition and they want to win.
Jensen: Saving them money, saving time. That’s really fantastic.
Building a Company Before the Market
Jensen: It’s such a great story. Of course, you started the company before the PC revolution really started. You must have written a business plan that pre-dated the PC revolution. Did you see it coming? How did you scope out the market?
Bob: I had a business plan. It was spiral-bound, on brown paper, and it was available on its first draft on September 30th, 1980—the same day as the Ethernet Blue Book—because my plans were predicated on the issuance of the Blue Book. The specs would be adequate to build a company, and they were.
I then began showing this business plan to people, and their first complaint was, “There’s no market research in your business plan.” My answer was, “This company that I’m proposing IS market research. If there’s a market out there, we’ll find it, and if there isn’t a market, we’ll go under.”
Jensen: I wish I had that line!
Bob: This sounds a little bit like “if you build it, they will come,” and I’m glad you came up with that phrase because one of the principles of Ethernet is: build it and they will come.
When we built that first Ethernet to run at 2.94 megabits per second, we did not have a requirements document from anybody saying they needed 2.94 megabits per second. We just built it as fast as the semiconductors would allow. That’s been a principle of Ethernet. They’re building an 800 gigabit per second Ethernet right now—I’m certain there’s no one requiring it. But “build it and they will come” has worked for the last 40 or 50 years.
Jensen: I still remember when we started the company, somebody said that to me too: “If you build it, they will come.” My response was, “If we don’t build it, they can’t come!” All innovators taking a chance, who believe in something—they have to build it based on dreams and hopes and confidence that there’s a world of innovation out there. So long as the platform could reach millions of people or millions of developers and smart people, somebody’s going to build something amazing at some point.
Breaking Through Shannon’s Law
Jensen: Most people think of Ethernet as networking technology. Of course, that networking technology is interoperable because it’s a standard. The technology, as you know well, has evolved tremendously since the beginning—since your 7400s and MSI chips and the specification of Manchester encoding. The physical layer, the data layer—the technology itself has evolved tremendously.
One of the things you said earlier: as far as semiconductor technology could enable it. In the case of Ethernet and switching technology, engineers are pushing the boundaries beyond semiconductor physics. In fact, Shannon’s Law was thrown at you in the early days when you were developing and seeing the evolution of Ethernet. Tell me about that moment. How did you react to it? Did you think that there was a physical limit that was very close? Did that concern you?
Bob: If you thought having IBM and General Motors attacking you was bad, wait till AT&T attacks you! Eighteen times, they attacked you with the law—with Shannon and information law.
The Bell Labs geniuses—and many of them were geniuses—said that the copper that had been put in by the telephone company for over 100 years connecting all those telephones had a Shannon limit of 14.4 kilobits per second. So all this talk about megabits per second wasn’t going to work over this pitiful copper because of Shannon’s Law.
Then one day someone started selling a 50 kilobit per second modem to run over the AT&T copper, and it worked. It worked just fine. I remember going to conferences rubbing AT&T’s nose in it. I said, “What about this Shannon limit thing that you guys have?”
Jensen: Do you remember FDDI versus CDDI? We had reached the limits of copper, and we had to go to fiber. Fiber was really expensive at the time, so the idea of 100 megabits per second was going to be too difficult to reach, too expensive. That company—the startup company that Cisco ultimately bought—came up with 100 megabits over copper and unleashed another several orders of magnitude of innovation after that.
Bob: That was Grand Junction Networks. I helped found that company with Howie Charney.
Lessons from Success and Failure
Jensen: You know, Bob, I have a theory about your company which I’d love to test on you right now.
Bob: This is not about me, Bob—this is about you!
Jensen: Give me a second! This is my time! Hundreds of thousands of people, millions of engineers are going to watch this. They came to watch you.
Let me get something out first. You and I, we’ve built companies, we’ve done a lot of things, we failed at a lot of things. Failures are great teachers—we learned a lot from failures. Failures are great teachers partly because of the emotional angst, the surprise, the disappointment that forces you to re-evaluate, to be introspective, and to do deep analysis to understand what happened. As engineers, we’ll always go back to first principles and recreate the model that predicted the first outcome and refine that model so we could predict future outcomes. Failure is a great teacher.
The thing that’s really extraordinary also is when something is a spectacular success—it’s also an excellent teacher. I’ve found that when you look at something that is an extraordinary, spectacular success, to be mindful of analyzing what were the dynamics and the events or circumstances that caused the extraordinary success. Some amount of serendipity helps, some amount of accidents help, but not really.
What were some of the lessons that you took that enabled Ethernet to be the magnificent success that it has become—the fabric that has connected the planet, that all of computing is built on today? What did you take away from that? What were some of the learnings that you would repeat when you start another company or that you would advise the young entrepreneurs you’re advising?
Bob: You almost said this, but let me say it: I think you learn more from success than from failure, because there are so many more ways to fail than there are ways to succeed. So if you find a way to succeed, you have found something of great value.
Ethernet was a 1973 invention, so it’s gone through a lot. The technologies have evolved, and almost every mistake you could make has been made. Let me try to recall a few.
In the building of 3Com, one of the mistakes we made a few times was to focus on our competitor instead of our customer. We went against Novell, which made PC networking software. We spun our wheels for two, three, four years fighting Novell—it was a complete waste of time.
Later, the year we went public, a startup started called Cisco. We didn’t give them enough respect. We didn’t pay attention to Cisco. It took them a long time, but they eventually blew past us in the ‘90s.
The mistake we made there was that they introduced routers with 14 protocols, and we had routers with two or three protocols. Being standards experts, we figured that you only needed three. But Cisco went out there and offered 14, and guess what? The customers wanted 14 protocols because they were filled with uncertainty about the future. They wanted to future-proof their acquisition. They wanted to be sure that no matter what protocol won, they would have it. That’s how Cisco slipped by us in the early fights over who would sell routers. So that’s a lesson.
NVIDIA’s Secret Weapon: Connectivity
Bob: Are you going to let me propose what I think about your company?
Jensen: There are so many entrepreneurs who would love to hear from you the lessons you’ve learned. You’ve dedicated the last decade of your career investing in companies, creating frameworks for innovation and for entrepreneurs to think about starting their company. I’m personally dying to hear about it, and I know there are millions of young entrepreneurs who would like to hear about it as well. So long as we can get back to that, I’ll answer any questions you have.
Bob: My theory of NVIDIA is that I go beyond the GPU. Clearly NVIDIA is the GPU company, but I don’t think that’s your secret. Your secret isn’t the GPU per se—it’s the interconnectivity among the GPUs. That’s your secret weapon. You’re in the same business I was in: connecting things together. I was in the business of connecting PCs and minicomputers together. You’re in the business of connecting GPUs together. What do you think of that theory of your company?
Jensen: The connectivity of small processors that are orchestrated and scheduled—micro-scheduling of a large network of processors—is very difficult. To take very complex programs, have them be distributed across all of these processors, have them be coordinated, communicate their intermediate results, and then get everybody going again—that orchestration, that scheduling is a very complex problem. It is the supercomputing problem, the large-scale massively parallel problem.
The technology is complicated from that perspective. People think that on-chip networks must be easier to do, and it is easier than the internet, of course. But the speed at which we communicate with thousands of processors, all able to talk to each other simultaneously—this fabric that we call it inside the chip, our internet if you will—is enormously complicated. It’s a machine in itself.
Tens of clock cycles are propagating across the fabric from thousands of processors all at the same time. We have to keep the latency down, the throughput high, and it has to be resilient to very large ranges of functional temperature, operating temperature, and process skews. We’re trying to keep everything down into the nanosecond, so the bit rate is really high. It’s a technology miracle.
That part of it is the giant breakthrough of a GPU, if you will. But that’s the technology. As you said earlier, customers don’t care about your technology—they care about their problem.
I would say that the wisdom of our company was to recognize that a GPU is a component, like a DRAM, like a CPU, like a network processor. It’s a technology component. How we formulate the entire solution to serve large domains of problems is really the breakthrough from a business perspective. It was a very challenging business model.
NVIDIA’s Vision: Accelerated Computing
Jensen: The way we articulated it was: look, there’s all these microprocessors moving at the speed of Moore’s Law. Everything we build next to it gets sucked into it. Everything we build next to it got sucked into it.
I still remember—our company was teed up as a business case in the early days. I wish I could see the presentation now, but it was presented to visiting professor Andy Grove. At the end of the business competition, we were described as a company that had no hope and would likely go out of business very soon.
The reason for that was a very good one. First of all, the articulated strategy of our company was to go solve problems—to solve challenging problems that are very large and sustainable, that have great impact in the world if solved, but that are not solvable by ordinary computers. Otherwise, you’d just code for CPUs.
If you’re choosing a problem that’s not solvable by CPUs, either that market doesn’t exist because the solution doesn’t exist, or that market is extremely small and it’s a supercomputer market. Neither of those conditions leads very well to a successful startup or a sustainable company. That was the articulated mission of our company: this is the problem we’re going to go solve.
The second problem is you sit next to the single most powerful black hole of technology in the history of humanity—everything near it gets sucked in. You could be traveling at the speed of light and still only be stationary next to it. The ability for a startup company to achieve R&D scale and not be sucked into the CPU because of Moore’s Law was probably close to impossible.
Between those two impossible conditions, Andy assessed that this company, though interesting, was not likely to succeed. And I would say the assessment was not wrong.
There was one piece—one observation, one insight. As you know, these market insights or strategy insights or innovation insights make a tremendous difference in how technology companies are ultimately formed.
The insight that we had was that the PC was coming, and it’s not likely that the most important killer app is recipes. It’s not likely that the killer app is spreadsheets. It’s important for office automation, but it’s not likely to be the application that puts a PC in everybody’s homes.
We came from the workstation industry, the semiconductor industry, so we knew very well what workstations can do. They were all connected by Ethernet. The killer app was a flight simulator where we’re chasing each other around the Bay Area, all sitting around laughing, chasing each other, having a great time because we’re suspended in this virtual reality world, networked together.
We theorized that someday the single largest consumer application is going to be virtual reality video games—3D games—and that everybody’s going to be a gamer. Who wouldn’t want to be a gamer?
This platform that we’re going to create would enable this new industry. At the time, I think Electronic Arts had like 12 employees. The 3D graphics video game market was completely non-existent. There were no standards—DirectX didn’t exist, OpenGL didn’t exist at the time. The PC had no sound, very poor graphics, didn’t even have joystick ports that allow you to interact with games.
But we imagined that someday the PC would evolve into this thing if we created this accelerated computing device that would enable these applications. The first application would be a video game. It could be the economic engine that gets the flywheel going, because this industry—video games—who wouldn’t want to play it?
The technology was so hard that the technology complexity times the size of the market would drive our R&D. That insight, that equation, has proven to be true.
This device that sits next to the CPU—the microprocessor that accelerates domains of applications that we have to go figure out, that we have to go evangelize, that we have to create the software stacks for—this approach to creating markets has proven to be quite a good one.
Now, as Moore’s Law has ended, we found ourselves in a time where this new era is coming along called AI. Computer science has found a new way of writing software, and this new way of writing software is a very specialized domain that requires tremendous acceleration that microprocessors can’t do. We found ourselves in the right place at the right time with the right set of skills, the right scale, the right perspective on how to invent and create new markets and new computing stacks.
I think it’s part insight, part technology differentiation, part strategy, part serendipity.
The Power of LAN Parties
Bob: I never went to one, but I’ve heard of things called LAN parties—local area network parties.
Jensen: Now I know what they were doing!
It’s really quite remarkable. The video game industry has not only become the largest entertainment industry in the world, it’s also used as a form of art. The graphics are now so beautiful you can mash them together to tell new stories. People use it for esports—it’s one of the fastest-growing sports. The people who play esports have incredible reflexes and snap-of-a-finger thinking and strategizing in real time. It’s like playing real-time multi-dimensional chess, and they’re incredibly good at it.
Now we’re seeing real sports coming into esports—F1 racing, NASCAR. The actual pro racers are sitting in these networked, Ethernet-worked pods that are racing each other, and they say it’s as intense as driving the real thing.
Video games, sitting on the fabric that you invented, makes it possible for us to network all of these gamers together. It absolutely follows Metcalfe’s Law, and Metcalfe’s Law has made all of these social games—massive online multiplayer games—which is built on a network, the reason why the video game industry is so large today. It is absolutely based on networks, network laws.
Standards for the Future
Bob: Today we are celebrating the standard, the first spec, the Ethernet Blue Book, September 1980. I argue that the standardization of Ethernet explains its success—by making it standard so multiple people could invest in it and multiple people could interconnect with it.
I’m wondering, is there now a standard that needs to be made in—or two standards or three—in gaming to advance gaming? Are there standards issues in gaming?
Jensen: No. I think you got it right. Standards make it possible for horizontal coopetition—between a lot of different players, you’re able to create a total solution for the marketplace that no single company can.
On the other hand, the standard of Ethernet was sufficiently high-level that it could be a foundation but not so high-level that innovation on top couldn’t happen. Therefore, the internet was built on top of it, and because the internet was built on top of it, now that’s a platform by which all kinds of applications are built on top of.
The brilliance of the Ethernet standard is that it’s really, to me, the ultimate platform strategy, the ultimate platform innovation. Sufficiently specified but sufficiently open-ended—that fine balance is what makes it possible for this rich ecosystem to have developed. I think it’s completely genius. Like poetry, it’s completely genius to figure out exactly what the words are and where to stop it. That genius has enabled all of these industries to happen.
The video game industry has specifications like DirectX and OpenGL, and those industry standards have absolutely enabled the video game industry to flourish.
The Next Era: The Internet of Things
Jensen: One of the things, Bob, that I want to talk to you about now is what’s next. The thing that’s really exciting—and we started the conversation about the announcement of ARM—is my interest in it and the reason why I think it’s going to be so important for the next era of internet.
The basis of my theory is this: the internet today is used largely by humans that are interacting with information and enjoying content and connecting with each other. Its value, the n-squared value—the n is humans, largely humans. It’s created trillions of dollars of economic value, accumulated over time.
The thing that I’m excited about is the confluence of a few technologies we’ve been working on—one particularly that’s going to activate a new internet. Of course, it’s an overlay, it’s a connection to this internet, but it’s really a new internet if you will: a new internet of autonomous machines.
These autonomous machines are going to be interacting with the internet, interacting with each other. Because of AI, you could write software for the very first time. Computers are just a box unless there’s software, and for the very first time we can write meaningful software that activates these computers at the edge to do interesting things—otherwise seemingly intelligent things like drive cars, do agriculture, keep cities and streets safe, move trucks around, robotic arms, and things like that.
That internet, unlike the current one we’re in, is not sporadic in the sense that we use the internet today sporadically. In the context of nanoseconds, we interact with a computer once every infinity, practically. But in the future, these machines are going to be interacting with the internet and interacting with each other continuously.
The size of the next internet we’re going to build is thousands of times bigger than the current internet we enjoy. That’s the reason why we bought ARM. Number one, we love their business model. Their business model makes it possible for companies far and wide—whether they’re in industrial equipment, manufacturing equipment, automotive, consumer electronics, whatever it is—for them to be able to build computers.
What we like to do is add to this network that ARM has, with AI computing technology that we’re very good at, to offer it to the ARM ecosystem so that they could build interesting computers using the same business model. By doing so, we’ll be able to automate, bring AI, make possible these autonomous machines to be intelligent and connect to the internet in a meaningful way.
We bought ARM for that reason: so that we could accelerate, if you will, bring AI computing to the ARM ecosystem. That’s the journey, really laid on the foundation that you laid. We would like to bring, we would like to enable, the next generation—the next chapter of the internet.
Bob: Would you call that the internet of things?
Jensen: Yeah, absolutely. It’s the internet of things.
I would say, Bob, that the internet of things has been floating in tech for practically a decade now, but we’ve just not been able to activate it. We’ve not been able to turn it on, per se. The reason for that is because the things need software, and the things oftentimes can’t be connected to the cloud. The reason for that is because they’re too chatty—they’re just talking continuously. You can’t put the autonomy, the autonomous software in the cloud. It’s just too expensive to do it. They’re supposed to be things and they’re chatty. They need control, they need to make decisions continuously.
We need to put, if you will, the data center that’s otherwise in a cloud—the software, artificial intelligence software—in the car itself. For example: autonomous driving cars, autonomous driving tractors, autonomous driving box movers, manufacturing arms, agriculture farming equipment, buoys out on the ocean, air quality measurement systems all over the world. Whatever it is, these things are so chatty you really want to find a way to put the computer closer to the edge. Oftentimes the data has sovereignty issues or privacy issues, or it’s impossible to move it to the cloud for whatever reason.
I think that AI is going to activate those things.
Bob: I was reading recently there are 7.7 billion humans on the internet, but there are 8 billion things on the internet already, even before what you’ve been talking about—bringing AI in for edge computing.
Jensen: That’s right. I think the internet is going to be thousands of times bigger than it is today. The amount of traffic that’s going to go through it is going to be just phenomenal.
Beyond Video: Transmitting Perception
Jensen: Some of the things that I’m particularly interested in, beyond that, in the field of computer vision and computer graphics, is to move beyond transmission of video as the way of being together.
Of course, today we transmit video, which is a two-dimensional image. But in the future, we’ll probably transmit perception, and from that information I can reconstruct you in 3D. You and I could be in the same space—basically a greater sensation of being together. Virtual presence is going to be possible.
Because of your invention today, we have the ability to share, to be in different places but share one digital space. In the future, we’ll teleport ourselves to different spaces. The teleporter is going to happen—it’s just going to be moving photons, not atoms.
The Art of Standards
Bob: To build all that and to trigger Metcalfe’s Law—I’m fond of calling it—we’re going to have to have standards. Carefully balanced, as you were saying earlier. They can’t be too low-level, they can’t be too high-level. You don’t need zero of them, you don’t need one. Maybe two or three will do, but ten standards is too many, generally speaking.
There’s this whole art of standards. Today we’re celebrating a very successful trade-off in that decision—in that standards-making, the Ethernet standard of 1980, which, by the way, bears no relationship to the Ethernet standard of 2020. There’s been a lot of evolution over the 50 years.
Jensen: Amazing work is built on top of your incredible invention. On behalf of all of the engineers and computer scientists in the world that participate in this industry, Metcalfe’s Law—this power law, this quadratic—has made it possible for us to enjoy an industry far greater than any of us could have possibly imagined in the early ‘90s. I thank you for that.
For all of the people who are watching, they must have learned a great deal from all of your body of work and your teachings and your previous online videos. I enjoy watching them, frankly. Your humor is incredible, and I want to thank you for the time you spent with me today. I hope I’ll have the pleasure of interviewing you some more in the future to tease out the body of knowledge that’s trapped in there for everybody else to enjoy.
Thank you very much. The contribution you made to computer science—as I mentioned, the technology is one thing, the wisdom about the standards another thing, the specification of that standard.
There’s a phrase in our company: “As much as needed, as little as possible.” That is the specification that I put on almost everything. I could find almost no application where that’s not true. It could be seasoning of steak—as much as necessary, as little as possible. Everything that I find—and people tend to do too much because they’re so clever, they want to contribute more, they want to control more. Whatever it is: more, more, more, more.
It is the genius that keeps it just enough. It is Steve Jobs’ genius not to add that last stroke. It’s Picasso’s genius not to add that last stroke. That genius requires discipline and vision and self-control and wisdom—all of that bottled up together. Ethernet did that, and you should be proud of that.
Bob: One of the slogans, like your slogan just then, that we had during the Ethernet standardization process was: “Anything that is not prohibited is mandatory.”
Jensen: [Laughs]
Bob: The idea there is that standards in the past have been destroyed by options. Some branch of the standard could use this option, others that. But in the Ethernet world, anything that was not prohibited was mandatory.
Jensen: It’s essential.
Bob: As minimum as possible, as little as possible, everything on that list therefore is essential.
Jensen: It’s really fun! Guys at Intel wanted the Ethernet to have a 16-bit hardware address, but we wanted a 48-bit address. So the IEEE standardized both 16 and 48. We had to build chips that could do both, but no one in the history of Ethernet has ever sent a packet with a 16-bit address.
Bob: It just never happened.
Jensen: I guess if you wanted to hook up a few peripherals—
Bob: Never happened.
East-West Traffic and the Future of Data Centers
Jensen: Incredible journey. I am genuinely excited about the next step. One of the things I wanted to talk to you about—something that I bet you and I wouldn’t have conceived of or even realized anyone would be crazy enough to do in computer science—I would have never thought this was going to happen.
We knew that a computer runs applications and that computer was going to get faster and faster, running bigger and bigger applications. Sometimes the application would be so large that you would connect a whole bunch of computers to run that application. That application, of course, at some point became a fairly complex application and runs in a virtualized environment—what is known as the data center.
This data center was used by enterprise, and then one day it moved out into the cloud. They were smart enough to build very large data centers that run consumer services using off-the-shelf but fully integrated servers—they call them hyper-converged infrastructures, otherwise became the word “hyperscale.”
The applications—one application would run in a server, and millions of people could be taken care of in the cloud. At some point, the applications became so different. Some of the applications are very complex, some are really just files—file retrieval. Some started to integrate artificial intelligence and speech recognition and image recognition and things like that.
The emergence of this new way of doing application composing is called microservices. Microservices are very efficient—just the bare minimum system resources to create a virtualized container. This container could be run anywhere in the data center, or this container will find the computers it could run in the data center. That second part is really important because the data center at this point is no longer homogeneous. It’s got some CPU, some storage server, some GPU-accelerated server.
You have an application with modules that are composed, and these modules are microservices that are running in different parts of the data center. I never thought that Ethernet is the connection between macros or modules within an application. That is pretty amazing.
Today’s data center, Bob, has these composable servers—what is called composable, disaggregated infrastructure. These microservices are orchestrated using this framework called Kubernetes. You could scale up and scale down, and the data center is fully utilized. The shock absorber—the system that’s making it possible—is the Ethernet switch, the Ethernet network. It’s putting enormous pressure on the networking. It’s now called east-west traffic, as you know very well.
Bob: Is this why you bought Mellanox?
Jensen: It is. It is exactly the reason why I bought Mellanox. It is exactly the reason why I bought Mellanox.
It’s always good to think about how to build your hardware by understanding software. Understanding software and understanding the direction of software inspires you about what’s the best way to design hardware and evolve hardware.
If you look at an application today, it used to be largely microprocessors and interactions with memory. Then it became microprocessors and GPUs and interaction with memory. Now it’s one computing node and interactions with another computing node, with the Ethernet network in between. Now it becomes the critical path of the data center.
If we could make the latency low—both by making the latency lower and by making the bandwidth higher—the throughput of data centers, the energy consumed in a data center, the efficiency of a data center, the cost of a data center would drop dramatically.
You and I were talking about earlier that customers, in the funnel analysis, want more for less. The cost of a data center would drop dramatically. We saw this coming, and it’s absolutely true. The east-west traffic in data centers is off the charts, and it’s going to keep on going off the charts.
Bob: Would you consider, or are you considering, Ethernet on chip?
Jensen: Ethernet on chip—it will probably be not necessary. The reason for not necessarily is because the protocols between our processors are very simple. They’re very simple, and we want to keep it as simple as possible. Everything that one processor should send to another processor is very well known, and so we could make a lot of assumptions about the sender and receiver, so the protocol doesn’t have to be nearly as sophisticated.
But inside racks, inside racks, it’s basically networking. In the future, Bob, we’re going to have an application that partly runs in a data center, partly runs in another data center at the edge, for example, and then partly that application is running on an autonomous machine roaming around the world. This way of composing applications connected by networking is fairly essential going forward.
Bob: That’s a grand vision.
Jensen: It’s pretty exciting: NVIDIA, Mellanox, ARM—we might be able to make something of this!
Bob: Yes! [Laughs]
Closing
Jensen: Bob, it’s such a pleasure. It’s such a great pleasure and such a great honor.
Bob: Thank you, Jensen.