Dan Weinreb’s blog

Don Piponi, who writes on the web under the nom de web “sigpfe”,  has written some amazingly clear papers on his blog, A Neighborhood of Infinity brilliantly explaining the elusive Haskell “Monad” concept. I’ll soon be blogging my own version of his stuff, with code in Lisp instead of Haskell.

I checked out his home page and found descriptions of some cute projects, including little home-made one-off robots that he has designed and created, called StirFry (autonomous motion avoiding obstacles), Equibot (balancing like a Segway), and OddJob (tiny autonomous robot that stops at black lines). There are fun videos of each. The links to the videos are at the bottom of each page.

He generously provides extensive instructions on how to make these, including some source code. I don’t know how hard real roboticists consider this, but from my point of view, as someone who has never done anything like this, it’s very impressive. Between this and the Monad papers (and others on his blog), this guy is so awesome!

TrI recently read How to Rig an Election: Confessions of a Republican Operative, by Allen Raymond and Ian Spiegelman” Raymond was a Republican operative, famously responsible for a denial-of-service attack on the Democratic HQ during a campaign in New Hampshire. (Speigelman is the ghost writer.)

In all fairness, if Raymond is telling the truth, most (but not all) of Republicans did not know he was doing this, and probably would not have sanctioned it. Given how harsh he is on the Republicans, I’m inclined to believe this claim.

The book is coyly written as if it were a mea culpa, whereas it’s also bragging. Of course he takes pot-shots at former colleagues, especially the ones who participated in the attack and left him hanging out to dry.

I’m not saying that Democrats are angels, but it seems to me that their unethical actions tend more towards putting friends on the payroll and such, rather than actually making our democratic/republican political system more corrupt.  The latter is far worse.  If more and more of our citizens lose faith in our political system, we lose social capital that is hard to regain.

Perhaps because I just read this book, I am predisposed to think that the following story from AlterNet is plausible: Conservatives have been gaming Digg:

First paragraph: “A group of influential conservative members of the behemoth social media site Digg.com have just been caught red-handed in a widespread campaign of censorship, having multiple accounts, upvote padding, and deliberately trying to ban progressives. An undercover investigation has exposed this effort, which has been in action for more than one year.”

If this be true, it’s alarming. Could it apply to other sites such as reddit (whose story about this has many of the screen names used by the Republican operatives)?

I was unable to find any corroboration; just lots of links the story, including on Digg). I also could not find any answer from Digg. If anyone reading this sees one, I’d be much obliged if you’d add a comment to this essay with a pointer thereto.

I was unable to find a reply at Digg’s web site, although I did find links to a bunch of very interesting stories. I tend to avoid Digg, reddit, et. al., because they are too good! I get distracted from what I’m really trying to do. Empty, casual browsing is so much less spiritually satisfying than browsing for deeper reasons.

Thanks to my wonderful friend and former co-worker Paul Harsha for bringing this to my attention.

Caveat: This story is changing in real time, so this essay may get obsolete quickly. Take a look at the comments to see if there are updates.

The New York Times, Bloomberg, and the Washington Post today said that Google and Verizon are talking about a deal in which Google/YouTube would pay Verizon to get more bandwidth for their own content. This is opposed to the principle of net neutrality, which Google has strongly supported for years, so this news is stunning, if true.

Verizon and Net Neutrality

Last October the CEO of Verizon made a speech about how net neutrality would be a bad thing, and would help Google et. al. at the expense of Verizon et. al. The key point, surprise surprise, is that he says net neutrality will hurt Verizon’s profit margin, who might take its ball and go home.

New York Times

Edward Wyatt of The New York Times today (8/5/2010) reports that Google and Verizon are working out terms so that Google/YouTube can pay Verizon to give them more “speed”/”priority”. Also fees for users from ISP’s will go up. His source is “People close to the negotiations who were not authorized to speak publicly about them.”

The article goes on to say: The FCC may squash this. Talks between the FCC and various companies continue. They are “jokingly” [my quotes] called the “secret meeting” [their quotes]. [Yeah, no kidding! It's surely what used to be called a "smoke-filled room".] Google, Verizon, AT&T, Skype, cable system operators, and the Open Internet Coalition, which has many members, including Amazon, ACLU, Am. Library Assn., ask.com, eBay, evite, Facebook, Google, match.com, NetFlix, PayPal, Skype, TicketMaster, TiVo, Twitter, and YouTube.

The Times story said nothing about who has how many lobbyists, who in Congress oversees the FCC, who contributes to the campaign of the committee members, and so on.

The first comment on the story simply said “Et Tu, Google?”, and more than one comment said “So much for ‘Don’t be Evil’.”

Wall Street Journal

The WSJ ran a story saying largely the same thing as the NY Times:

Bloomberg

Todd Shields (tshields3@bloomberg.net) at Bloomberg ran a related story:

Verizon Communications Inc. and Google Inc. have struck their own accord on handling Internet traffic, as both participate in talks by U.S. officials on Web policy, two people briefed by the companies said.

We’ve been working with Google for 10 months to reach an agreement on broadband policy,” said David Fish, a Verizon spokesman. “We are currently engaged in and committed to the negotiation process led by the FCC.”

Google has “nothing to announce at this point,” said Mistique Cano, a Washington-based spokeswoman, in an e-mail.

They also point out that announcing Android phone availability has helped Verizon earnings a lot.

Their quote in favor of net neutrality comes from Andrew Jay Schwartzman of the Media Access Project, a public-interest law firm.

Verizon and Mountain View, California-based Google proposed in a January filing at the FCC areas of compromise for regulating Internet service providers. The companies said preserving an “open Internet” calls for “minimal interference from the government” for applications, content and services, such as Google and Twitter.

Google

Google issued the following denial, at about 10:30am EDT:

@NYTimes is wrong. We’ve not had any convos with VZN about paying for carriage of our traffic. We remain committed to an open internet.

On Gizmondo they macroexpanded this from tweet-ese to English:

The NYT is quite simply wrong. We have not had any conversations with Verizon about paying for carriage of Google or YouTube traffic. We remain as committed as we always have been to an open Internet.

Google has been on record as supporting net neutrality for years.

A Google/Verizon deal of the kind described by the New York Times would enact precisely the pay tiers that Schmidt fiercely fought in 2006. Jeff Jarvis calls Google’s agreement a “devil’s pact with Verizon for tiered internet service.” Huffington Post blogger and Free Press president Josh Silver warns, “The deal marks the beginning of the end of the Internet as you know it.”

This may in fact be just the latest crack in Google’s support for net neutrality. The Wall Street Journal reported in 2008 that Google was approaching broadband providers in the hopes of creating a “fast lane” for its own content.

In Jan 2010, Google came out strongy in favor of net neutrality.

Net Neutrality Background

Back in Nov 2005, Vint Cerf, one of the creators of the Internet, now the Chief Internet Evangelist at Google, testified in absentia (he was receiving the Presidential Medal of Freedom at the White House!), to Representatives aJoe Barton and John Dingell, then of the Committee on Energy and Commerce in the House, strongnly in favor of net neutrality. In Oct 2009, he was also interviewed in the Washington Post supporting net neutrality.

He co-wrote an open letter to Julius Genachowski, the FCC Chairman. The net neutrality proposal wasn’t public yet, but the ISP’s were fighting against it. The point isn’t to disallow traffic shaping at the packet level, but to “prevent anti-compeitive practices.” It’s interesting that they used that particular phrase, which comes from the Sherman anti-trust act. The letter was signed:

• Vinton G. Cerf, Internet Pioneer
• Stephen D. Crocker, Internet Pioneer
• David P. Reed, Internet Pioneer
• Lauren Weinstein, Internet Pioneer
• Daniel Lynch, Internet Pioneer

The FCC issued the “Notice of Proposed Rulemaking” very shortly thereafter, and then voted with 3 in favor, and 2 in favor of some but not the rest. ArsTechnica analyzes what they voted for, and shows ambiguities and exceptions. However, this ruling was made null and void, by the Comcast decision.

Comcast Decision

Comcast was ordered by the FCC to stop throttling traffic from P2P services such as BitTorrent. I can see how they might not like this given that they are charging a fixed fee for bandwidth.

Gizmondo says:

Never mind the huge conflict of interest, that most of them are also trying to sell you video services through your TV, so they’d prefer you not watch it for free on your computer, or that Comcast is now buying the majority of one of those networks who produces content for your TV.

For instance, totally hypothetically, now that Comcast, via its NBC shares, owns a chunk of Hulu, it could give network priority to Hulu over Netflix Watch Instantly streaming. Or Microsoft could pay Comcast to give Zune Video priority over another service on the network.

Also, Comcast is trying to buy NBC Universal, which includes NBC and several cable channels. Would it favor its own content? So far, they say, surprise surprise, that they won’t. To me, that means they won’t do it until they do it.

The FCC sued to prevent Comcast from interfering with its customers’ use of peer-to-peer networking applications. In April 2010, the case reached a court one step away from the US Supreme Court, namely the US Court of Appeals for the D.C. Circuit. They decided to make a broad ruling (rather than one narrowly-tailored to this case) that the FCC does not have “authority to regulate an Internet service provider’s network management practices.”

The Court vacated the order, and granted Comcast’s petition for review. This means that Congress would have to pass a law to give the FCC these powers. Until then, the FCC is powerless to order Comcast and friends to stop throttling P2P sites.

The National Cable and Telecommunications Association stated that the court ruled that the FCC decision “was wrong”. This is highly misleading: what they ruled is, right or wrong, the FCC doesn’t have the power to do this. The Court’s decision is not specious; it is quite plausible.

But, because the ruling was so broad that the FCC cannot mandate net neutrality either, unless new laws are passed! I worry that once non-neutral charging is in place, imposing net neutrality could be much harder, so the delay could be harmful (in my non-expert opinion)

A New York times article about the decision points out that “some conservative Republicans” will be against giving more power to the FCC. That’s a nice, polite way of putting it; I’d be very surprised if more than one or two Republicans vote to give the FCC any new powser, and the Democratic “majority” is fractured into micro-interests rather than working together. The article continues:

As a practical matter, the court ruling will not have any immediate impact on Internet users, since Comcast and other large Internet providers are not currently restricting specific types of Web content and have no plans to do so.

Until they do.

Worst Case Scenario

Without net neutrality, your Internet bill could look like your cable TV bill, with extra monthly fees for optional services like movies, news, international news, web search, and finally a big fee for full Internet access. Of course, the carriers say that they have “no intention” of doing this. If they did have an intention to do it, or were even considering it, of course they’d say that they’re not doing so. Why give the enemy advance notice of your attack?

The only thing we know for sure that they’re doing is thinking about how to make more money. There’s nothing wrong with making as much money as you can, legally. But it’s up to society (ultimately, the voters) to decide what’s legal: what are the rules of the game that are in the public interest. They’ll try hard to change the rules. The power of the political donor class, and their lobbyists, is beyond the scope of this essay. But I hope that we won’t end up with this.

Losing Net Neurtrality: The Worst Case Scenario (from Gizmondo)

I recently was reminded that  Air New Zealand’s airline reservation system went out of service on Sunday morning at about 9:30 am, October 11, 2009.  This story is very interesting to me, since my team is building just such a system (with a very different underlying implementation).

IBM said that the outage happened because of a power failure at an IBM data center in Newton that took out their mainframe.  Many existing airline reservation systems run on a single IBM mainframe; mainframes are known for being rock-solid reliable, but not without electricity! IBM said it was caused by a failed oil pressure sensor on a backup generator.  What’s more, the problem happened during a scheduled maintenance session!

The outage affected more than 10,000 passengers, leaving airports “in disarray”.  Most systems were restored around 1.30 pm [four hours later], but the passenger backlog did not start to clear until self check-in kiosks were up and running again about 3.30pm [six hours later]. Air New Zealand was, to put it mildly, furious.

As usual, people never (well, hardly ever) adequately test their redundant backup technology! In particular, they should have also used the generators for a long enough time to test for this kind of failure.  I recently heard about another such pr0blem, in a discussion at ITA, where the backup generators worked but didn’t work for long enough.  (At least I think it was a distinct case, since I heard it a while ago, but I could be wrong.)

You must do these tests reasonably frequently, since things can break over time, even if they are merely lying in wait. I plan to write more about this in a future blog post.

I don’t know, but I’ve been told: most, if not all, airlines do not actually have disaster recovery setup that would switch over to a geographically distant site.  Evidently airlines are surprisingly “penny wise and pound foolish” when it comes to redundant components, which they are loath to pay for.  (I think we were talking about network connections but it’s too long ago for me to remember clearly.  The same principle applies across the board.)

Afterward, apparently IBM’s main job was to grovel.  Air New Zealand, in the person of CEO Rob Fyfe, said in strong language that IBM took a long time to react, accept responsibility, and apologize.  He called IBM “amateur”, which is quite an insult for IBM, and that his IT team was looking for alternative suppliers already.  (I don’t know how that turned out.)

IBM did apologize by the evening of the next day, and said they “immediately engaged a team of 32 local IT professionals, supported by global colleagues. This means Mr. Fyfe considers two working days to be a very long time for such an apology.  Perhaps he was manly putting on a public show of anger, actually intended more for his customers and shareholders than for IBM.  But I don’t think that actually matters, from IBM’s point of view.  As someone participating in a team building such a system, that’s the point of view that I am most concerned with.

(By the way, back at MIT in the late 1970′s, when a guy from Digital Equipment showed up for “preventive maintenance” on one of our timesharing systems (removable disks on the MIT-MC KL/10), we called it “causitive maintenance”.  He once made a mistake that caused a lot of trouble.)

I don’t just mean this to rag on IBM.  Making systems that are working fully 24/7 is quite difficult and expensive.  Our team can perhaps learn a little bit from this: if nothing else, it is one more data point about the cost/benefit of system failure for an airline reservation system.  When airlines talk about high availability, they are not kidding!

I just upgraded from Thunderbird 2 to Thunderbird 3 on my Vista machine, and I like it a lot. If you haven’t heard about version 3, I recommend that you upgrade. I have not yet tried it on my work machine, running the old version 8 (“Intrepid”) of Ubuntu; if you have experience with this, I’d appreciate hearing about it.

Some cool things:

Search is much faster, due to indexing. (When I first brought up version 3, it spend a long time indexing, but that’s because I have a huge number of saved messages. I was able to use it while it indexed.)

When you select several messages, it displays extended summaries of all of them: whom it’s from, date, and the first few lines.

If you double-click on a message in the summary line, instead of creating a new window, it creates a “tab” in the main Thunderbird window, a lot like Firefox tabs.

The operations on a message being read are now down where the message is, to the right of the header display. This seems a bit more intuitive, although I don’t think it matters much.

Archive: The new “Archive” command moves the message into a folder. The folder is the subfolder of a new top-level folder called “Archive”. The subfolder’s name is the year from the date of the message.

If you use the word “attach” in a message you’re composing, it brings up a fairly unobtrusive bar a the bottom saying “Found an attachment keyword: attached” or whatever word you used (you can control the set of keywords), followed by buttons for “Add Attachment…” or “Remind me later”. When I used Thunderbird 2, I had an addon called “Attachment Reminder” that would pop up a menu when you clicked “Send”, asking you if it was OK. That’s more obtrusive, which you might or might not like.

One bad thing:

There’s no more “Forward all as attachments”, which let you send a message that contained many attachments, namely everything you had selected in the summary pane. I have used that many times.

My favorite add-on is “Expression Search”, which provides a powerful and concise way to search. The best thing is that it saves you from having to use the mouse to select the type of search to do (the “magnifying glass” dropdown in the upper-right search bar).

I also like ConfirmFolderMove (helps prevent accidental “move folder” operations.

(I’m just starting to try MailClassfier, which would be very helpful if it turns out to work. But the first time I tried it, it was analyzing all of my mail, and at the end of this slow process, Thunderbird crashed. And when I restarted it asked again to analyze all mail, so apparently the results were not saved before the crash. I’ll keep trying to make it work.)