The simple answer is yes. I personally fixed dozens of instances of it. Most of them were trivial and would have been at worst an annoyance, but they were real, and a few would have been of real consequence. Particularly serious was that the Microsoft BASIC string parser defaulted to years beginning with 19xx and was used in lots of user and database interactions, including pretty much anything related to OLE. I fixed a bunch of these issues during my brief time in the BASIC team in the mid '90s, and lots of other people did too. MS-DOS prior to 5.0 had the bug. The only 5 inch floppy drive I still have is on a machine running MS-DOS 4.0, and I have to lie to it about the date in order to read such a floppy.
The scariest bug I know about is that apparently a bunch of spy satellites were not working properly for several days immediately following 31 Dec 1999. DoD kept quiet about this for several months, until they were sure they'd gotten it fixed.
I was pretty worried about infrastructure problems: computer controlled water supply, banking, traffic controls, etc. I set aside a reserve of cash, charcoal and camp stove fuel, and about 20 gallons of drinking water, just in case. Fortunately, none of this was necessary...but Jan 1 turned out to be a fairly nice day here and I had a nice barbeque... Lots of people had fixed lots of bugs. For example, I heard about a hydroelectric project that had experimented with their fix in June, lying to the dam controls about the date and watching what happened as the date clicked over.
Not sure if my (then) new car with a lot of computer controls would work, I used my old car to drive to my friend's place for a new years party. It had no computers aboard at all. But on the trip home, I crossed the 520 bridge at about 1AM. The bridge was open, and was for a long time. So I got out and walked to the drawspan. The workman there said it was scheduled maintenance, nothing to worry about. Yeah, right. Middle of the night on New Years Day. I am confident that this was in fact a Y2K bug but you won't get anyone in DoT to admit it.
No really bad things happened on Y2K day, but that was because there'd been a LOT of worrying.
Here's a list of problems that one group studying this found. Unfortunately, the "official" page they reference has gone away. One estimate was that $100B was spent fixing instances of it. I suspect this is high...I worked on Y2K problems off and on for several years...If you imagine that I spent the entire time working on Y2K, you'd get an accounting of several hundred $K, but that misstates it. Over several years, I spent much less than a month cumulative time. But tens of thousands of programmers did something similar.
Take the example of the hydroelectric dam. The programmer involved needs to do whatever software changes are deemed necessary, and test them to the limit of their ability without actually affecting the water system. Lets say 5 man days. Then they need to staff up the whole facility so if something bad happens, they can manually override, almost no matter what it is. Once everybody is ready, they install the new software, lie to it about the date, and watch what happens as the date ticks over. Probably a dozen people for a day, or perhaps more if something goes wrong the first time. we're up to almost 20 man days. Repeat this tens of thousands of times for thousands of different types of installations, all around the world.
Yes, Y2K was a problem. One which was dealt with almost adequately.
Showing posts with label computers. Show all posts
Showing posts with label computers. Show all posts
16 February 2015
23 September 2014
Date Formats
There are quite a few formats in common use for presenting the date. The most common in the US is
MM-DD-YY, which represents today's date as
September 23, 2014, and can be abbreviated as Sep 23, 14 or even 9/23/14
In Europe, the most common is
DD-MM-YY, which represents today's date as
23 September 2014 and be similarly abbreviated
Some people also use the "big endian" version of this
YY-MM-DD, which comes out
2014 September 23, which has a certain appeal. (This is ISO 8601)
as long as everybody is using the same format, it doesn't really matter that much which we use. But too often, you can't tell. After a lot of thinking about it, I've concluded that little endian euro style is the way to go. the biggest advantage is that it requires no punctuation or even spaces to remain readable:
23Sep14 is almost as decipherable as 23 September 2014 and requires only 7 bytes to represent all possible 21st century dates. (it takes 3 letters to disambiguate March from May and June from July.) by adding two more to indicate the century, 23Sep2014 is completely unambiguous, self documenting, (no Y2K problem!) and can represent any date from the reign of Augustus to 31Dec9999 in 9 human readable bytes. If for some reason I choose to write it 2014Sep23, you can still read it correctly.
If I write it Sep2314, you can still probably figure it out, but it could also be 14 Sep 2023. If I go really crazy and write "91101", you aren't really sure it's a date at all--it could be a zip code (it happens to be Pasadena, CA). Or it could be 1 Jan 1991, 9 Nov 2001, or 11 Sep 2001.
So, unless there's a good reason to do otherwise, I use Euro little endian when I write the date: 23 Sep 2014.
MM-DD-YY, which represents today's date as
September 23, 2014, and can be abbreviated as Sep 23, 14 or even 9/23/14
In Europe, the most common is
DD-MM-YY, which represents today's date as
23 September 2014 and be similarly abbreviated
Some people also use the "big endian" version of this
YY-MM-DD, which comes out
2014 September 23, which has a certain appeal. (This is ISO 8601)
as long as everybody is using the same format, it doesn't really matter that much which we use. But too often, you can't tell. After a lot of thinking about it, I've concluded that little endian euro style is the way to go. the biggest advantage is that it requires no punctuation or even spaces to remain readable:
23Sep14 is almost as decipherable as 23 September 2014 and requires only 7 bytes to represent all possible 21st century dates. (it takes 3 letters to disambiguate March from May and June from July.) by adding two more to indicate the century, 23Sep2014 is completely unambiguous, self documenting, (no Y2K problem!) and can represent any date from the reign of Augustus to 31Dec9999 in 9 human readable bytes. If for some reason I choose to write it 2014Sep23, you can still read it correctly.
If I write it Sep2314, you can still probably figure it out, but it could also be 14 Sep 2023. If I go really crazy and write "91101", you aren't really sure it's a date at all--it could be a zip code (it happens to be Pasadena, CA). Or it could be 1 Jan 1991, 9 Nov 2001, or 11 Sep 2001.
So, unless there's a good reason to do otherwise, I use Euro little endian when I write the date: 23 Sep 2014.
11 August 2014
Bitcoin and Gold
Many people think that gold is a good currency because it has intrinsic value. This is almost (not quite) complete nonsense. Nothing has intrinsic value, except for goods and services. The value of a thing is what people are willing to buy and sell it for. If it keeps its value, it's because somebody is willing to pay.
Back in pre-history, people traded things with each other. This is sometimes inconvenient--the buyer may not have what the seller wanted, and a third party, who does have what the seller wanted and also wants what the buyer has, might be necessary. Fairly early on, people figured out that instead of a third party, a third item which had a fairly reliable trade value, because almost everybody found it useful, could increase the possibility of negotiating such trades. Items which could easily be adjusted to fit the value needed were more useful--grain, pieces of cloth, etc. Things which are easy to carry around are also better--and of course the more rare the item, the more valuable it is, which makes it easier to carry around. The reason gold rose to the top of this heap is because it's both rare and very difficult to fake: it's heavier than any more common metal (19.32 gm/ml. Lead is 11.34.) and it's soft and doesn't tarnish. That's it. But that's very useful, and before long, nearly every trader or laborer would happily take gold for their goods or services. Somebody, most likely in ancient Lydia (today, part of western Turkey) figured out how to make standardized coins with a press around 700 BC, which is also about the time the Bible was written, and this idea quickly became universal. Coins made of baser metals were less valuable, both because they were more common than gold, and because they were easier to fake. But they were still convenient. Even then, none of these necessarily maintained their value--inflation happened with gold as with any other currency, although what was happening was not understood until the 15th century.
Some people, including Ron Paul and Lyndon LaRouche and a lot of equally confused people, think that there's something special about gold. There is, but it's only that scarcity and difficulty of forging. Anything else that has a limited supply and is difficult to forge works. Fiat currency--printed money--was invented about a thousand years ago in China. This is very convenient--government can print as much as it needs. But it didn't take long for people to figure out how to fake it. And if government printed too much, inflation got out of hand and destabilized the economy. Central banks were created to manipulate the money supply. Today, most of the world money supply is just numbers in account books or their computer equivalent. The central banks of the world manipulate demand for it and thus the amount in circulation by manipulating interest rates.
Bitcoin is no different: it's just a form of currency that's difficult to fake and the supply is constrained by a technological trick, which impedes the printing of it. The appeal for some people is that because government is not involved, government won't be able to manipulate it. This is true, and it's a bad thing. Since, like coins and printed currency, it's very hard to trace, it's been a boon for criminals looking to hide or launder their transactions. And because it's hard to produce much, if demand skyrockets, its price will skyrocket too.
There are very few cases of the Federal Reserve and other central banks doing things to devaluate our money, in fact quite the reverse: in 1979-81 the fed intentionally drove up interest rates and caused substantial unemployment to stop inflation. Had a sizable part of the economy been out of their control through a mechanism like bitcoin, both inflation and unemployment would have been much higher.
There have been cases of government explicitly devaluating money, e.g. the 1967 devaluation of the British Pound, and there have been lots of cases of incompetent, corrupt governments printing lots of money and causing hyperinflation. But as long as the Federal Reserve is relatively immune from political pressure from economic naïfs like Ron and Rand Paul, and professionally staffed, it's unlikely to do that.
Back in pre-history, people traded things with each other. This is sometimes inconvenient--the buyer may not have what the seller wanted, and a third party, who does have what the seller wanted and also wants what the buyer has, might be necessary. Fairly early on, people figured out that instead of a third party, a third item which had a fairly reliable trade value, because almost everybody found it useful, could increase the possibility of negotiating such trades. Items which could easily be adjusted to fit the value needed were more useful--grain, pieces of cloth, etc. Things which are easy to carry around are also better--and of course the more rare the item, the more valuable it is, which makes it easier to carry around. The reason gold rose to the top of this heap is because it's both rare and very difficult to fake: it's heavier than any more common metal (19.32 gm/ml. Lead is 11.34.) and it's soft and doesn't tarnish. That's it. But that's very useful, and before long, nearly every trader or laborer would happily take gold for their goods or services. Somebody, most likely in ancient Lydia (today, part of western Turkey) figured out how to make standardized coins with a press around 700 BC, which is also about the time the Bible was written, and this idea quickly became universal. Coins made of baser metals were less valuable, both because they were more common than gold, and because they were easier to fake. But they were still convenient. Even then, none of these necessarily maintained their value--inflation happened with gold as with any other currency, although what was happening was not understood until the 15th century.
Some people, including Ron Paul and Lyndon LaRouche and a lot of equally confused people, think that there's something special about gold. There is, but it's only that scarcity and difficulty of forging. Anything else that has a limited supply and is difficult to forge works. Fiat currency--printed money--was invented about a thousand years ago in China. This is very convenient--government can print as much as it needs. But it didn't take long for people to figure out how to fake it. And if government printed too much, inflation got out of hand and destabilized the economy. Central banks were created to manipulate the money supply. Today, most of the world money supply is just numbers in account books or their computer equivalent. The central banks of the world manipulate demand for it and thus the amount in circulation by manipulating interest rates.
Bitcoin is no different: it's just a form of currency that's difficult to fake and the supply is constrained by a technological trick, which impedes the printing of it. The appeal for some people is that because government is not involved, government won't be able to manipulate it. This is true, and it's a bad thing. Since, like coins and printed currency, it's very hard to trace, it's been a boon for criminals looking to hide or launder their transactions. And because it's hard to produce much, if demand skyrockets, its price will skyrocket too.
There are very few cases of the Federal Reserve and other central banks doing things to devaluate our money, in fact quite the reverse: in 1979-81 the fed intentionally drove up interest rates and caused substantial unemployment to stop inflation. Had a sizable part of the economy been out of their control through a mechanism like bitcoin, both inflation and unemployment would have been much higher.
There have been cases of government explicitly devaluating money, e.g. the 1967 devaluation of the British Pound, and there have been lots of cases of incompetent, corrupt governments printing lots of money and causing hyperinflation. But as long as the Federal Reserve is relatively immune from political pressure from economic naïfs like Ron and Rand Paul, and professionally staffed, it's unlikely to do that.
10 August 2014
A Few Things I'd Rather Not See
Facebook:
The message "viewing top stories. Back to Top Stories". I have never once wanted "Top Stories", although I occasionally look at it to see what it was suggesting. I always went right back in about 5 seconds.
Trending posts.
Suggested posts
Other people's achievements in on-line games.
Amazon and other ecommerce sites offering to post about my purchases.
I would like the ability to sort by user: a few folks are extremely noisy and they swamp more interesting posts. Perhaps a view that limits any one poster to three a day.
I'd also like a way of categorizing posts: it's very rare that I want to see pictures of people's meals, games, or cats, but occasionally people who post a lot of these have something worthwhile to say. FB is disturbingly good figuring out the general topic of a message....let me explicitly say up or down to the category.
IPhone:
The store popping up when I least expected it. I occasionally buy things, but I always chose the store rather than fumbling into it, with no clear way to get back, when the UI doesn't provide an obvious way to do what I'm looking for. This is especially common in the ipod mode.
I have never once hidden traffic on purpose in the new (since v7) map. yet about every fifth time I use the app I find traffic turned off. it's potentially useful to do if it's obscuring road names or something. (I would like a quick way to get roadnames to become big enough I can see them without my reading glasses)
there are many, many apps that have a very important button which is too close to other buttons to touch without a stylus. Fitbit's iPhone "dashboard" puts "go to the next day", which is about 40% of my use of the app, between "edit", which I have never used, and "device status", which I use about 1% of the time.
Windows8
The stupid windows button everywhere, where I constantly hit it when I was aiming for ctrl, or gripping my mouse or something. there are 4 of them in my current view. one would be ample.
please please please, give me back the XP interface and all that came with it, such as the ability to kill off apps when I'm done with them.
I've been using Win8 for several months now, and I've found exactly two improvements: better control of the start menu cache: it's called "Metro", and touch support works. regressions in amazingly many other areas.
petty but important: the new "xbox" card games are basically unplayable with a mouse, they are a pain to get to, and they frequently ask me if I'd like to send them money for stupid crap I don't want. I'll have to try the xp versions.
The message "viewing top stories. Back to Top Stories". I have never once wanted "Top Stories", although I occasionally look at it to see what it was suggesting. I always went right back in about 5 seconds.
Trending posts.
Suggested posts
Other people's achievements in on-line games.
Amazon and other ecommerce sites offering to post about my purchases.
I would like the ability to sort by user: a few folks are extremely noisy and they swamp more interesting posts. Perhaps a view that limits any one poster to three a day.
I'd also like a way of categorizing posts: it's very rare that I want to see pictures of people's meals, games, or cats, but occasionally people who post a lot of these have something worthwhile to say. FB is disturbingly good figuring out the general topic of a message....let me explicitly say up or down to the category.
IPhone:
The store popping up when I least expected it. I occasionally buy things, but I always chose the store rather than fumbling into it, with no clear way to get back, when the UI doesn't provide an obvious way to do what I'm looking for. This is especially common in the ipod mode.
I have never once hidden traffic on purpose in the new (since v7) map. yet about every fifth time I use the app I find traffic turned off. it's potentially useful to do if it's obscuring road names or something. (I would like a quick way to get roadnames to become big enough I can see them without my reading glasses)
there are many, many apps that have a very important button which is too close to other buttons to touch without a stylus. Fitbit's iPhone "dashboard" puts "go to the next day", which is about 40% of my use of the app, between "edit", which I have never used, and "device status", which I use about 1% of the time.
Windows8
The stupid windows button everywhere, where I constantly hit it when I was aiming for ctrl, or gripping my mouse or something. there are 4 of them in my current view. one would be ample.
please please please, give me back the XP interface and all that came with it, such as the ability to kill off apps when I'm done with them.
I've been using Win8 for several months now, and I've found exactly two improvements: better control of the start menu cache: it's called "Metro", and touch support works. regressions in amazingly many other areas.
petty but important: the new "xbox" card games are basically unplayable with a mouse, they are a pain to get to, and they frequently ask me if I'd like to send them money for stupid crap I don't want. I'll have to try the xp versions.
10 August 2013
Vaporware
I just read the wikipedia article on Vaporware, and tracked down a few of its references. Interestingly, the inventor of the term is purported to be an engineer for Microsoft Xenix in 1982. There are only about 8 people which that title describes and I'm one of them. The ultimate reference is this 1995 article in the New York Times, which describes a 1982 meeting Ann Winblad had with Mark Ursino and John Ulett. I knew Ursino and Ulett quite well and I think I know what the meeting was about, although I wasn't there. First of all, they're what we called in those days "Marketeers", which was a job that included sales, marketing and what later came to be called "program management". They have some technical understanding, but they were not engineers. For a purely OEM product like Xenix, their job mostly consisted of talking to folks--media like Winblad1, Dyson, etc., potential customers, existing customers, etc., and ultimately the engineers themselves. If I understand the context, what U&U were trying to say with the vaporware comment was that while we intended to do what we'd promised, but there was not an engineer actually working on it at that particular moment. After all, there were only 8 of us and hundreds of promises made. We did eventually stop working on Xenix, but it was not until 1987. We earnestly did try to do everything we'd committed to. We did a lot--Xenix for 8086, z8000, 68000, 286, 386, and in most cases, several wildly divergent platforms for each, and several versions of Xenix. The specific thing I'm guessing Winblad was concerned about, the ability to make an atomic database operation in the face of file system caching and several processes having the file open simultaneously, was done in early '83, about a year after the relevant meeting. It wasn't terribly hard, but it took more than a week.
In the 1960s and '70s, IBM made a determined effort to capture the entire computer market. They came quite close to pulling it off. One of the many dirty tricks they would play was to announce and attempt to sell a bunch of products that somebody thought might be useful, and see which ones had the most buyers. Once they found out, they'd set about implementing those few, not bothering to implement the majority of the proposed products. A smaller company wouldn't have the working capital to pull this off. A customer considering choosing a competitor can easily be reigned in, for the cost of some hype. Eventually, this, and many other of IBMs monopolist stunts, would be banned, but not before most of their competitors had failed. This strategy is now known as "selling vaporware".
It seems to me there are three legitimate things which might be called vaporware:
4. We don't intend to do this, but we're pretending we are, to inhibit you from going with our competitor.
In other words, lying to the customer. Unfortunately, too many people on the marketing side think this is ok. If you change jobs every year or two and never have to face the consequences of being caught in a lie, this may be ok for you. But it undermines the future of the organization that appears to have done it.
1 Winblad was at the time still with a company doing accounting software, Open Systems, Inc, but her real importance was the articles she was writing.
In the 1960s and '70s, IBM made a determined effort to capture the entire computer market. They came quite close to pulling it off. One of the many dirty tricks they would play was to announce and attempt to sell a bunch of products that somebody thought might be useful, and see which ones had the most buyers. Once they found out, they'd set about implementing those few, not bothering to implement the majority of the proposed products. A smaller company wouldn't have the working capital to pull this off. A customer considering choosing a competitor can easily be reigned in, for the cost of some hype. Eventually, this, and many other of IBMs monopolist stunts, would be banned, but not before most of their competitors had failed. This strategy is now known as "selling vaporware".
It seems to me there are three legitimate things which might be called vaporware:
- We're working on it, but it'll take some time before it's ready.
- We intend to work on it, but it'll be a while before we can even free up enough time to start.
- We don't really intend to work on it unless we get enough customer demand.
4. We don't intend to do this, but we're pretending we are, to inhibit you from going with our competitor.
In other words, lying to the customer. Unfortunately, too many people on the marketing side think this is ok. If you change jobs every year or two and never have to face the consequences of being caught in a lie, this may be ok for you. But it undermines the future of the organization that appears to have done it.
1 Winblad was at the time still with a company doing accounting software, Open Systems, Inc, but her real importance was the articles she was writing.
25 July 2012
Al Gore Invented the Internet!
Well, not really, and he didn't say he did either. But he did play a crucial role.
The internet is a collection of protocols--TCP/IP, DNA, HTTP, RFC-822, lots more. Nearly all of them were invented by, or under the aegis of, some US or European research grant. The most important of these came from the US Defense Department's Advanced Research Projects Agency, known as DARPA or ARPA. A number of computer networks were invented in the late 60s, but nearly all of them were fundamentally centralized. DoD saw great value in the networking, but realized that centralized networks were extremely susceptible to attacks on the critical links. At the time, the Cold War and the risk of Nuclear Attack was at the top of DoD's mind. Two ARPA researchers, Vinton Cerf and Bob Kahn, realized that a highly decentralized network would be immune to this problem, and designed some rudimentary protocols, which they called NCP, and commissioned some other researchers to begin implementing them. It didn't take long to come up with something useful, and BBN (Bolt Beranek and Newman) built the first router under DARPA contract, which they called an IMP (Interface Message Processor)
The nascent network, called the ARPANet, became immediately popular for academic and military. ARPA, which was footing the entire bill, quickly put their foot down and banned commercial traffic lest the bill get too large. Since it was fundamentally a research network that had become far more popular than expected, they'd included no billing mechanism at all. At the same time, a number of other companies were selling various other types of networks. The most popular were Bulletin Board Systems and Local Area Nets. Nearly all of these used the Star organization--a central hub with all communication passing through it, although there were a few decentralized networks that grew up, such as UUCPNet, and a few LANs that used token passing. Token passing is decentralized, but limited in scale, and Star is not decentralized--to make it bigger, the central hub needs to be bigger and pretty quickly becomes an unaffordable bottleneck. But the completely decentralized ARPANet did not need any big hubs. IMPs got cheap fast, and a lot of people quickly realized that service providers could be small and independent and they could all work together, making a whole that was much larger than the sum of the parts. But there was a problem of how to charge the users, to make the thing scalable and commercial.
Cerf and Kahn and a few others set about solving the problem, and came up with a new set of protocols, which they called TCP/IP (Transmission Control Protocol/Internet Protocol) which improved a lot of things, and included some sourcing information, which allowed service providers to implement billing. Since the ARPANet had been around for over ten years and was very popular by this point, it took a lot of politicking to impose this change, and it was Al Gore who was the champion of it in congress. (I was one of the skeptics at the time. I was wrong. Cerf was right) He got the bill passed, and on New Years Day 1983, called "Flag Day" for internet users, NCP was abandoned and TCP/IP replaced it. The internet was born. At the same time Domain Name Architecture or System (DNS...the obvious acronym would be confusing) was imposed.
Over time, a number of competing protocols have been in play. Some of them, such as X25, are potentially superior, but none have been able to compete with the incredible flexibility and installed base of TCP/IP. All of the Star/Hub networks have been forced to accommodate the internet or go out of business. Many are still in use as LANs or ISPs, but the majority have switched to using TCP/IP. During the late 80s/early 90s, a great many BBS-like networks grew up, including Compu$erve, MSN, Prodigy, AOL, and more. All of these used the Star architecture. It made billing simple and and forced users to mainly use content of the providers choice, which of course created more profit centers. Each had incompatible ways of generating content, although they began to focus on what were called Markup Languages and HyperLinks. On the ARPANet, you had to have some level of sophistication about files and data in order to use things. the network providers provided, among other things, a high level of user-friendliness. They used HyperText and so did most of the non-network CD-ROM content, such as HyperCard and Microsoft Encarta.
HyperText had been invented in the mid 70s by a brilliant eccentric named Ted Nelson. His idea was that all the information in the world could be brought together and linked, by placing them all on a centralized set of computers and putting pointers in the various documents to one another. He saw there being a centralized editorial board to make sure all the content was correct and consistent. Nelson's idea was consistent with the Star/Hub idea. He wrote a fun book about it and lots of other things, called Computer Lib/Dream Machines in 1974 which caught the imagination of thousands of computer scientists. One of them was Tim Berners-Lee, an Englishman and researcher at CERN, the European Nuclear Research Center (paid for by a consortium of European governments). Berners-Lee realized that by modifying a markup language and making a browser to use it, and providing a simple internet protocol layer, which he respectively called HyperText Markup Language and HyperText Transfer Protocol (HTML and HTTP), he could create a version of Ted Nelson's vision. His motive was to provide a flexible host for CERN researchers to document their experiments, apparatus, and more, but it didn't take long before he realized that everybody could use it. He called it the World Wide Web.
The internet is a collection of protocols--TCP/IP, DNA, HTTP, RFC-822, lots more. Nearly all of them were invented by, or under the aegis of, some US or European research grant. The most important of these came from the US Defense Department's Advanced Research Projects Agency, known as DARPA or ARPA. A number of computer networks were invented in the late 60s, but nearly all of them were fundamentally centralized. DoD saw great value in the networking, but realized that centralized networks were extremely susceptible to attacks on the critical links. At the time, the Cold War and the risk of Nuclear Attack was at the top of DoD's mind. Two ARPA researchers, Vinton Cerf and Bob Kahn, realized that a highly decentralized network would be immune to this problem, and designed some rudimentary protocols, which they called NCP, and commissioned some other researchers to begin implementing them. It didn't take long to come up with something useful, and BBN (Bolt Beranek and Newman) built the first router under DARPA contract, which they called an IMP (Interface Message Processor)
The nascent network, called the ARPANet, became immediately popular for academic and military. ARPA, which was footing the entire bill, quickly put their foot down and banned commercial traffic lest the bill get too large. Since it was fundamentally a research network that had become far more popular than expected, they'd included no billing mechanism at all. At the same time, a number of other companies were selling various other types of networks. The most popular were Bulletin Board Systems and Local Area Nets. Nearly all of these used the Star organization--a central hub with all communication passing through it, although there were a few decentralized networks that grew up, such as UUCPNet, and a few LANs that used token passing. Token passing is decentralized, but limited in scale, and Star is not decentralized--to make it bigger, the central hub needs to be bigger and pretty quickly becomes an unaffordable bottleneck. But the completely decentralized ARPANet did not need any big hubs. IMPs got cheap fast, and a lot of people quickly realized that service providers could be small and independent and they could all work together, making a whole that was much larger than the sum of the parts. But there was a problem of how to charge the users, to make the thing scalable and commercial.
Cerf and Kahn and a few others set about solving the problem, and came up with a new set of protocols, which they called TCP/IP (Transmission Control Protocol/Internet Protocol) which improved a lot of things, and included some sourcing information, which allowed service providers to implement billing. Since the ARPANet had been around for over ten years and was very popular by this point, it took a lot of politicking to impose this change, and it was Al Gore who was the champion of it in congress. (I was one of the skeptics at the time. I was wrong. Cerf was right) He got the bill passed, and on New Years Day 1983, called "Flag Day" for internet users, NCP was abandoned and TCP/IP replaced it. The internet was born. At the same time Domain Name Architecture or System (DNS...the obvious acronym would be confusing) was imposed.
Over time, a number of competing protocols have been in play. Some of them, such as X25, are potentially superior, but none have been able to compete with the incredible flexibility and installed base of TCP/IP. All of the Star/Hub networks have been forced to accommodate the internet or go out of business. Many are still in use as LANs or ISPs, but the majority have switched to using TCP/IP. During the late 80s/early 90s, a great many BBS-like networks grew up, including Compu$erve, MSN, Prodigy, AOL, and more. All of these used the Star architecture. It made billing simple and and forced users to mainly use content of the providers choice, which of course created more profit centers. Each had incompatible ways of generating content, although they began to focus on what were called Markup Languages and HyperLinks. On the ARPANet, you had to have some level of sophistication about files and data in order to use things. the network providers provided, among other things, a high level of user-friendliness. They used HyperText and so did most of the non-network CD-ROM content, such as HyperCard and Microsoft Encarta.
HyperText had been invented in the mid 70s by a brilliant eccentric named Ted Nelson. His idea was that all the information in the world could be brought together and linked, by placing them all on a centralized set of computers and putting pointers in the various documents to one another. He saw there being a centralized editorial board to make sure all the content was correct and consistent. Nelson's idea was consistent with the Star/Hub idea. He wrote a fun book about it and lots of other things, called Computer Lib/Dream Machines in 1974 which caught the imagination of thousands of computer scientists. One of them was Tim Berners-Lee, an Englishman and researcher at CERN, the European Nuclear Research Center (paid for by a consortium of European governments). Berners-Lee realized that by modifying a markup language and making a browser to use it, and providing a simple internet protocol layer, which he respectively called HyperText Markup Language and HyperText Transfer Protocol (HTML and HTTP), he could create a version of Ted Nelson's vision. His motive was to provide a flexible host for CERN researchers to document their experiments, apparatus, and more, but it didn't take long before he realized that everybody could use it. He called it the World Wide Web.
29 November 2011
The Falacy of Very High Level Languages
During my career as a systems programmer, I interviewed hundreds of candidates and was intimately involved with dozens of development projects, from one man teams to teams of hundreds, writing in binary, assembler, C, and various "high" level languages, and a few very high level languages,. Over the course of all of this one of the things I realized was that language tools, such as big libraries, extremely high levels of abstraction, object orientation, and quite a few other "tools" are at best crutches, which can actually get in the way of solving the problem. During the 70s and 80s, a bunch of people noticed that developers writing in assembler had a much higher success rate than developers working in BASIC or several other high level languages. Without thinking very carefully, they blamed the particular high level language--for example, lack of structured programming tools. But that's not the problem.
I spent quite a bit of time thinking about this. What's happening, I think, is that the various tools promote barriers to entry.
On this chart, every programmer/development environment has a point. A poor programmer is toward (or off) the bottom, better ones toward the top. Working in a lower level language puts you toward the left, higher level toward the right, as exemplified by a few languages listed across the top. As everybody knows, working in a lower level language makes it harder to get a program working than in a higher level language. I think there are actually two thresholds at work. The lower threshold, the lower, blue line, is the "can get it limping" level. If you're below that level, you'll never get anything to work at all. As you can see, a lot more programmers can get something going at the higher level languages than at the lower level.
The higher, red line, is the "good program" level. If a programmer is working below the red line, they'll never produce something good. The red line is at a very shallow slope for a simple reason: The things that make a good program--critical thinking and analysis, memory, good planning, deep grasp of the problem and effective use of approaches to solving it--are largely independent of programming language or style. If your thinking is muddled, you won't do a good job, irrespective of language, tools, design approach, or whatever.
The key, and the reason the "best" programmers seemed to be working in the hardest language, is that assembler provides such a high barrier to getting anything working at all that most of the muddled thinkers can't ever get anything that even looks like it's working. There's not much space between the red and blue lines when you're working in assembler. The lower level languages allow more programmers to look like they're producing something useful, while not actually doing much. 90% of the productive work is done by the 5 or 10% that are above the red line.
Up until about 1990, a programmer's ability to work in assembler was a pretty good calibration of their overall competence. A good assembler programmer working in a higher level language would simply be more productive. But after that time, astonishingly few professional programmers really understood the machine. By 1997, it was rare to find a entry level candidate for an operating systems team that had more than a smattering of assembly experience. And the higher level languages allow a lot more muddled thinkers into the process. In the 70s and 80s, we couldn't tolerate much muddled code into the final product--there was some, but memory and disk space was too precious. Nowadays, nobody seems to care. The muddled thinking--and remember that with today's very high level languages, which functionally consist of gluing libraries together, that's an extremely high percentage--becomes a very high percentage of the shipping project. One consequence of this is that you must download and install 700MB to update your telephone. If all the programmers involved were working above the red line, it would probably be about 20MB.
I spent quite a bit of time thinking about this. What's happening, I think, is that the various tools promote barriers to entry.
The higher, red line, is the "good program" level. If a programmer is working below the red line, they'll never produce something good. The red line is at a very shallow slope for a simple reason: The things that make a good program--critical thinking and analysis, memory, good planning, deep grasp of the problem and effective use of approaches to solving it--are largely independent of programming language or style. If your thinking is muddled, you won't do a good job, irrespective of language, tools, design approach, or whatever.
The key, and the reason the "best" programmers seemed to be working in the hardest language, is that assembler provides such a high barrier to getting anything working at all that most of the muddled thinkers can't ever get anything that even looks like it's working. There's not much space between the red and blue lines when you're working in assembler. The lower level languages allow more programmers to look like they're producing something useful, while not actually doing much. 90% of the productive work is done by the 5 or 10% that are above the red line.
Up until about 1990, a programmer's ability to work in assembler was a pretty good calibration of their overall competence. A good assembler programmer working in a higher level language would simply be more productive. But after that time, astonishingly few professional programmers really understood the machine. By 1997, it was rare to find a entry level candidate for an operating systems team that had more than a smattering of assembly experience. And the higher level languages allow a lot more muddled thinkers into the process. In the 70s and 80s, we couldn't tolerate much muddled code into the final product--there was some, but memory and disk space was too precious. Nowadays, nobody seems to care. The muddled thinking--and remember that with today's very high level languages, which functionally consist of gluing libraries together, that's an extremely high percentage--becomes a very high percentage of the shipping project. One consequence of this is that you must download and install 700MB to update your telephone. If all the programmers involved were working above the red line, it would probably be about 20MB.
28 July 2011
Reinventing and Glue
Engineers are prone to something called NIH--Not Invented Here--because that is what engineers like to do: figure out how to do something and implement it. They'd like to "Reinvent the world". That's hyperbole, of course, but it's a tendency that their managers try to get them to resist: If somebody has already done it, the current team doesn't need to do any new work and chances are good that the previous, ostensibly successful implementers, did a better job than the current team would. By and large, this is good advice. But not always.
A lot of the time, the old implementation doesn't quite exactly do what the new one needs to. To repurpose the old work, we have to invent a new interface, to be able to fit it into the new work. In software, we call that "glue code", or just "glue". It's rare that glue is simple. If the old work was well designed and modularized, it's at least a few lines of code. It's usually quite a lot more than that, and surprisingly often, it turns out to be more than the code that's being reused. The code that's being reused doesn't have to be tested, says management. Perhaps, but the glue does have to be tested just as hard, and since the fit isn't perfect, the old code does have to be tested. Suppose the old component was 2000 lines long. If the glue is anything up to, say, 300 lines or so and the function really is similar, it's probably worth trying to make the glue work. But if the glue is more than a thousand lines, you're really getting into diminishing returns. It's very likely that reinventing the thing from scratch will work better for the new application, and even if it's 2000 lines long, you've eliminated the need for glue, saving a thousand lines and a bunch of testing.
There are other cases too: sometimes the old implementation wasn't all that well done. Fred Brooks, in The Mythical Man Month, talks about second systems syndrome. When doing the first implementation, it's all they could do to get it to work at all, and they likely were changing algorithms and interfaces all along. It's quite likely the first implementation is kludgey and not too good. The second time, they know better, but now they have a whole bunch of new ideas and they get a whole lot of bloat. On the third try, they're getting closer to the Goldilocks point: not too big nor too little, refined, shaken down, appropriate algorithms, well thought out interfaces. If the code you're trying to reuse is a first or second implementation, it's very likely you're just propagating a bad thing.
Finally, there are a lot of things that are more appropriate being reimplemented. For example, simple searches or insertions that occur at user time. This is such a simple algorithm that nearly everyone who has written the code has done it several times, and won't get it wrong. A new implementation will fit the new codebase perfectly and avoid any mingled source-tree complications. Even if uses totally naive algorithms, it's often better to be simple than optimal, and it's likely that there's no actual algorithmic advantage to be gained (e.g: a bubble sort is so much simpler that it's actually faster than NlogN sorts if there are fewer than a dozen or so things to be sorted)
A lot of the time, the old implementation doesn't quite exactly do what the new one needs to. To repurpose the old work, we have to invent a new interface, to be able to fit it into the new work. In software, we call that "glue code", or just "glue". It's rare that glue is simple. If the old work was well designed and modularized, it's at least a few lines of code. It's usually quite a lot more than that, and surprisingly often, it turns out to be more than the code that's being reused. The code that's being reused doesn't have to be tested, says management. Perhaps, but the glue does have to be tested just as hard, and since the fit isn't perfect, the old code does have to be tested. Suppose the old component was 2000 lines long. If the glue is anything up to, say, 300 lines or so and the function really is similar, it's probably worth trying to make the glue work. But if the glue is more than a thousand lines, you're really getting into diminishing returns. It's very likely that reinventing the thing from scratch will work better for the new application, and even if it's 2000 lines long, you've eliminated the need for glue, saving a thousand lines and a bunch of testing.
There are other cases too: sometimes the old implementation wasn't all that well done. Fred Brooks, in The Mythical Man Month, talks about second systems syndrome. When doing the first implementation, it's all they could do to get it to work at all, and they likely were changing algorithms and interfaces all along. It's quite likely the first implementation is kludgey and not too good. The second time, they know better, but now they have a whole bunch of new ideas and they get a whole lot of bloat. On the third try, they're getting closer to the Goldilocks point: not too big nor too little, refined, shaken down, appropriate algorithms, well thought out interfaces. If the code you're trying to reuse is a first or second implementation, it's very likely you're just propagating a bad thing.
Finally, there are a lot of things that are more appropriate being reimplemented. For example, simple searches or insertions that occur at user time. This is such a simple algorithm that nearly everyone who has written the code has done it several times, and won't get it wrong. A new implementation will fit the new codebase perfectly and avoid any mingled source-tree complications. Even if uses totally naive algorithms, it's often better to be simple than optimal, and it's likely that there's no actual algorithmic advantage to be gained (e.g: a bubble sort is so much simpler that it's actually faster than NlogN sorts if there are fewer than a dozen or so things to be sorted)
22 July 2011
Display Resolution and Aspect Ratio
In 1994, the "Digital HDTV Grand Alliance" decided upon a new television standard for the whole world. They specified a list of resolutions, most of which have an aspect ratio of 16x9. They chose to allow various aspect ratios for individual pixels, and they did not specify any resolutions past 1920x1080. These definitions were hugely counterproductive, not just for the TV industry but also for the then rapidly growing personal computer industry, as well as being confusing for most users, and completely, utterly, unnecessary.
Between the invention of moving pictures and about 1960, nearly all video had an aspect ratio of 4x3. There was good reason for this: Many things in our environment, from buildings, to sporting events and conversation groups, fit well in the 4x3 frame. When television came along, the movie business felt threatened and in their effort to come up with a more immersive experience, they invented wider aspect ratios. Because of the Cathode Ray Tube display technology in use at the time, TV could not duplicate this. Many movies, but by no means all, embraced the new mode and placed subjects wide apart in the frame. When those movies came to be displayed on 4x3 screens, the incompatibilities were addressed with two bad options: "Pan and Scan", which loses the width of the widescreen cinematography, and "Letterbox", which uses only a small part of the screen to show the whole picture, but in a much reduced window. The movie people were understandably upset by damage done to their cinematography...a problem which was entirely of their own creation.
When digital TV became possible, the same people saw a way out of the box into which they'd painted themselves: they'd make the new TVs support widescreen. These same people managed to dominate the conversation. Never mind that movies are only a small part of TV content and that 4x3 worked very well for almost everything else. Never mind that the new digital technology made it trivial and transparent to support any resolution you chose to transmit in. They knew analog TV hardware and movies, and made a standard with that mindset and only the next 5 years or so in mind. They didn't understand Moore's law, or really anything about computers at all, even those these were central to what they were doing.
What they should have done was standardize two things: the digital data format and square pixels. Period. They should specifically NOT have specified resolution. If I want to watch a lot of movies, I'll buy a display with a wide aspect ratio. If I'm a sportsfan or TV news addict, I'll buy a display with a low aspect ratio. If I want to do the other once in a while, I'll suffer in just exactly the way I have been for movies. But it'll be MY choice, and I'll choose the style that serves me best. The technology to adapt easily is a necessary part of every digital TV display (and personal computer). There's no need to standardize.
The free market is a powerful thing. When higher resolution displays become available, the market will decide which resolutions to sell, not some committee that's stuck in the mistakes they made in the 1950s. The software and hardware to adapt content to match the display (it's called stretching and cropping) is a necessary part of every digital TV and computer display controller. Plugging in a different set of numbers is simple. Adapting content resolution to style, cost and availability of the necessary bandwidth is equally simple. If half the country is watching the superbowl or some major movie, it makes sense to use a lot of bandwidth. If I'm watching a talking head discuss current events, I don't mind if my picture is being transmitted at 320x240. Save the money the bandwidth costs.
It turns out that most people have a hard time understanding resolution and aspect ratio. I'm not sure why this simple thing should be so hard, but it is. But because the HD Alliance chose to provide for both 1:1 and 3:4 pixels, they dramatically increased the confusion. When 4x3 content with 1:1 pixels appear, a great many TVs stretch it to wide aspect ratio, so as to fill the screen--distorting the picture. Moreover, the manufacturers tend to bury the necessary mode controls--I suppose because usability testing found them confusing, and consequently amazingly many people have gotten so used to watching distorted pictures they've stopped noticing. By simply standardizing on 1:1 pixels, the TV could automatically adapt. No user controls would be necessary. Instead, we have various and confusing "wide" "panorama", "4x3", "Zoom", etc., modes.
HD aspect ratio sickness has infected computers, too. In 2003 I bought a laptop with a 1400x1050 (so called SXGA+) display surface. Today, it's fairly challenging to find a laptop with a vertical resolution higher than 768, 73% the size of that 8 year old computer. This is because the market for display surfaces is completely dominated by the TV business--just as in the days of CRTs--and most small TVs are 1280x720. You can buy a monitor with much higher resolution: up to 2560x1600, but even finding out what the resolution of most display surfaces is from the marketing literature or salescritter is surprisingly difficult. I care MUCH less how big it is in inches, than I do how much VERTICAL resolution it has. Nearly all applications and websites have wide menu bands across the top and bottom, dramatically reducing the space available (the window I'm using to type this subtracts 350 pixels from the top of my window and 160 from the bottom--leaving only 258 pixels of usable space on a 768 high monitor. Windows 7 live photo gallery (the app that provoked this flame) steals 200 pixels from the top and 95 from the bottom when in editing mode, 38%. what could be more important in a photo app than being able to get the picture as big as possible?)
Between the invention of moving pictures and about 1960, nearly all video had an aspect ratio of 4x3. There was good reason for this: Many things in our environment, from buildings, to sporting events and conversation groups, fit well in the 4x3 frame. When television came along, the movie business felt threatened and in their effort to come up with a more immersive experience, they invented wider aspect ratios. Because of the Cathode Ray Tube display technology in use at the time, TV could not duplicate this. Many movies, but by no means all, embraced the new mode and placed subjects wide apart in the frame. When those movies came to be displayed on 4x3 screens, the incompatibilities were addressed with two bad options: "Pan and Scan", which loses the width of the widescreen cinematography, and "Letterbox", which uses only a small part of the screen to show the whole picture, but in a much reduced window. The movie people were understandably upset by damage done to their cinematography...a problem which was entirely of their own creation.
When digital TV became possible, the same people saw a way out of the box into which they'd painted themselves: they'd make the new TVs support widescreen. These same people managed to dominate the conversation. Never mind that movies are only a small part of TV content and that 4x3 worked very well for almost everything else. Never mind that the new digital technology made it trivial and transparent to support any resolution you chose to transmit in. They knew analog TV hardware and movies, and made a standard with that mindset and only the next 5 years or so in mind. They didn't understand Moore's law, or really anything about computers at all, even those these were central to what they were doing.
What they should have done was standardize two things: the digital data format and square pixels. Period. They should specifically NOT have specified resolution. If I want to watch a lot of movies, I'll buy a display with a wide aspect ratio. If I'm a sportsfan or TV news addict, I'll buy a display with a low aspect ratio. If I want to do the other once in a while, I'll suffer in just exactly the way I have been for movies. But it'll be MY choice, and I'll choose the style that serves me best. The technology to adapt easily is a necessary part of every digital TV display (and personal computer). There's no need to standardize.
The free market is a powerful thing. When higher resolution displays become available, the market will decide which resolutions to sell, not some committee that's stuck in the mistakes they made in the 1950s. The software and hardware to adapt content to match the display (it's called stretching and cropping) is a necessary part of every digital TV and computer display controller. Plugging in a different set of numbers is simple. Adapting content resolution to style, cost and availability of the necessary bandwidth is equally simple. If half the country is watching the superbowl or some major movie, it makes sense to use a lot of bandwidth. If I'm watching a talking head discuss current events, I don't mind if my picture is being transmitted at 320x240. Save the money the bandwidth costs.
It turns out that most people have a hard time understanding resolution and aspect ratio. I'm not sure why this simple thing should be so hard, but it is. But because the HD Alliance chose to provide for both 1:1 and 3:4 pixels, they dramatically increased the confusion. When 4x3 content with 1:1 pixels appear, a great many TVs stretch it to wide aspect ratio, so as to fill the screen--distorting the picture. Moreover, the manufacturers tend to bury the necessary mode controls--I suppose because usability testing found them confusing, and consequently amazingly many people have gotten so used to watching distorted pictures they've stopped noticing. By simply standardizing on 1:1 pixels, the TV could automatically adapt. No user controls would be necessary. Instead, we have various and confusing "wide" "panorama", "4x3", "Zoom", etc., modes.
HD aspect ratio sickness has infected computers, too. In 2003 I bought a laptop with a 1400x1050 (so called SXGA+) display surface. Today, it's fairly challenging to find a laptop with a vertical resolution higher than 768, 73% the size of that 8 year old computer. This is because the market for display surfaces is completely dominated by the TV business--just as in the days of CRTs--and most small TVs are 1280x720. You can buy a monitor with much higher resolution: up to 2560x1600, but even finding out what the resolution of most display surfaces is from the marketing literature or salescritter is surprisingly difficult. I care MUCH less how big it is in inches, than I do how much VERTICAL resolution it has. Nearly all applications and websites have wide menu bands across the top and bottom, dramatically reducing the space available (the window I'm using to type this subtracts 350 pixels from the top of my window and 160 from the bottom--leaving only 258 pixels of usable space on a 768 high monitor. Windows 7 live photo gallery (the app that provoked this flame) steals 200 pixels from the top and 95 from the bottom when in editing mode, 38%. what could be more important in a photo app than being able to get the picture as big as possible?)
Subscribe to:
Posts (Atom)