The router explanation really got me thinking about how much invisible infastructure is working every time we click a link. I never considered the whole routing protocol where routers ask each other about hops until finding the shortest path. And submarine cables spanning oceans just to delver a tweet is wild. Do you think most people would use the internet differently if they understood all this complexity?
I love these kinds of posts! While not having any official credentials in software development or planning (my background is a MechE specializing in fluid dynamics), I managed a web development group at NASA for 20 years and am now a planning commissioner in Annapolis, so those backgrounds seem to go together, especially in the Strong Towns world where methodologies like agile software development and incremental development strongly rhyme.
Anyway, it is indeed a miracle. I have always been amazed at how much the "founding fathers" initially got right with the LAN/WAN technology stack and how well it's scaled - that was the idea after all! - as computing power has grown exponentially. I'm old enough to remember the message there were "no circuits available" in emergencies or busy calling days before voice traffic moved from circuit switch to packet switched networks. About the only criticism I can make about the development of the internet is that security is an afterthought layered on top. It's a testament to the flexibility of technology that this actually works reasonably well now and we shouldn't be revisionist about that now as at the time they neither had the appreciation for the future scale at which this would be used nor the computing power to implement security from the ground up. It's surprising to me none of the original designers have ever won a Noble Prize.
The routing protcol explanation really stood out to me. Most people think packets just magically find their way across the internet, but the way routers recursively query each other to find the shortest path is genuinly brilliant. That plus the submarine cables spanning oceans makes you realize how much phsyical infrastructure we're relying on every time we load a page. Wild to think all that complexity happens in milliseconds.
The walkthrough of how routers actually find each other by asking neighbors about hops was eye-opening. I've worked in backend dev but never realy visualized that rippling search happening in real-time across continets. Also appreciated that you didnt gloss over cellular networks and Cloudflare, even though you skipped the deep dive. What part of this stack do you think people misunderstand the most when talking about internet access issues?
Hmm. Mostly I think people don’t understand the implications of a distributed and packet switched network. So, they imagine that loading a web page is like walking up to a house and knocking on its door, and then the person coming out and handing you a book. But its actually more like passing notes from person to person across a classroom to ask the person on the other side of the room for the book, and then they cut the pages out of the book and send them back one page at a time, and then you put the pages back together! The point being that many things can and do go wrong within the request/response cycle itself that are not the fault of the software at either end.
We’ve made the internet so good that junior developers treat API calls like normal function calls and often don’t realize how fundamentally different they are, therefore don’t think about the fact that the network calls sprinkled through their code actually will fail sometimes :)
Interesting. I used to be a software engineer but I retired and got a degree in urban planning.
i’m a software engineer thinking about possibly getting into urban planning haha. why is this so common
The router explanation really got me thinking about how much invisible infastructure is working every time we click a link. I never considered the whole routing protocol where routers ask each other about hops until finding the shortest path. And submarine cables spanning oceans just to delver a tweet is wild. Do you think most people would use the internet differently if they understood all this complexity?
I love these kinds of posts! While not having any official credentials in software development or planning (my background is a MechE specializing in fluid dynamics), I managed a web development group at NASA for 20 years and am now a planning commissioner in Annapolis, so those backgrounds seem to go together, especially in the Strong Towns world where methodologies like agile software development and incremental development strongly rhyme.
Anyway, it is indeed a miracle. I have always been amazed at how much the "founding fathers" initially got right with the LAN/WAN technology stack and how well it's scaled - that was the idea after all! - as computing power has grown exponentially. I'm old enough to remember the message there were "no circuits available" in emergencies or busy calling days before voice traffic moved from circuit switch to packet switched networks. About the only criticism I can make about the development of the internet is that security is an afterthought layered on top. It's a testament to the flexibility of technology that this actually works reasonably well now and we shouldn't be revisionist about that now as at the time they neither had the appreciation for the future scale at which this would be used nor the computing power to implement security from the ground up. It's surprising to me none of the original designers have ever won a Noble Prize.
Totally agree! Thanks for sharing, Alex!
Grateful for it and for Substack
The routing protcol explanation really stood out to me. Most people think packets just magically find their way across the internet, but the way routers recursively query each other to find the shortest path is genuinly brilliant. That plus the submarine cables spanning oceans makes you realize how much phsyical infrastructure we're relying on every time we load a page. Wild to think all that complexity happens in milliseconds.
The walkthrough of how routers actually find each other by asking neighbors about hops was eye-opening. I've worked in backend dev but never realy visualized that rippling search happening in real-time across continets. Also appreciated that you didnt gloss over cellular networks and Cloudflare, even though you skipped the deep dive. What part of this stack do you think people misunderstand the most when talking about internet access issues?
Hmm. Mostly I think people don’t understand the implications of a distributed and packet switched network. So, they imagine that loading a web page is like walking up to a house and knocking on its door, and then the person coming out and handing you a book. But its actually more like passing notes from person to person across a classroom to ask the person on the other side of the room for the book, and then they cut the pages out of the book and send them back one page at a time, and then you put the pages back together! The point being that many things can and do go wrong within the request/response cycle itself that are not the fault of the software at either end.
We’ve made the internet so good that junior developers treat API calls like normal function calls and often don’t realize how fundamentally different they are, therefore don’t think about the fact that the network calls sprinkled through their code actually will fail sometimes :)
Great stuff, subscribed!