This debate was central to the fascinating discussion at the State of Telecom event at Columbia’s Instititue of Tele-Information, held in mid-October. I attended the afternoon sessions.

Wall Street’s vision

Simon Flannery, managing director at Morgan Stanley, described the challenges that are eroding the margins of the cellcos. Of course, the top two cellcos are doing better than the rest. Flannery said that margins at Verizon at about 45 percent, while margins at Sprint are about 16 percent. Apps that are eroding core revenues include free text messaging and free calling.

Meanwhile, cellcos are selling advanced services that require more bandwidth. “Backhauling fiber to towers requires a massive build,” said Flannery. “Smaller carriers lack the cash flow to reinvest, and there is no financing for newtowkrs that are without returns.”

The market is trending towards a duopoly.

Craig Moffett, senior analyst at Bernstein Research, said that the services that people pay for are the easiest to provide: internet access, phone calls, and so on. “People are less willing to pay for information and entertainment, which are services that cost more to provide.”

Voice may require 9.6 Kbps and people will pay $50 per month for it. People will only pay about $30 per month more for the next generation services that multiply data usage by 10 or 100 times. “The sale price per bit is falling faster than the cost per bit.”

At the bottom end of the market, Moffett said, there is the “poverty problem” where households whose net income is negative after paying for food, clothing, and shelter account for perhaps 40 percent of all homes. “Retailers can depend on the upper two quintiles, but telcos have to sell to the full 100 percent of the population,” Moffett claimed.

The consumer advocate concurs

Mark Cooper of the Conumer Federation claimed that he disagreed with everyone on all sides of this debate. He said that in some poor countries, there are 75 cell phones per 100 people. “People who have no electric power at home may have cell phones.”

If a cellco is just adding voice customers, it’s easy to grow. “It is easy to add subscribers but it is expensive to add capacity. Users, uses, and usage all add costs to wireless mobile networks.”

He agreed with the Wall Street analysts that the marginal sale price of bandwidth drops rapidly.

He added, however, that unlicensed wireless spectrum is the great success story of the past two decades. Even AT&T is now selling Wi-Fi. “Unlicensed has no champion in the scrum for spectrum.” Cooper said that at most 10 percent to 20 percent of spectrum should be sold to the cellco monopolies, so that the Washington can avoid the next monopoly and the next “100 year mistake.”

The Economist Magazine has the data

In an article entitled The Limits of Frugality, The Economist magazine warns that those rural cellphone users who have no electricity in their homes will soon be paying higher prices. “Sunil Mittal, the boss of Bharti Airtel, the mobile-phone operator … said the extra cost of servicing rural customers, and their low usage levels, had made things unprofitable. Prices are now expected to go up across the industry, after two decades of decline. India’s low-cost telecoms revolution has, it seems, reached its limit.”

Buildouts will now focus on the urban rich. “Today perhaps 17 percent of India’s population has half of its spending power, according to the Asian Development Bank …. One proxy for the difference in profitability between the urban rich and the rural poor is the price paid for mobile-telecoms spectrum. In the 2010 auctions for 3G telecoms licences, operators bid ten times more for a slice of the airwaves in affluent Delhi, with 18m people, than in east Uttar Pradesh, with 120m people.”

Washington, D.C.’s policymakers should expect fixed wireless and wireline internet to connect the rural poor at an affordable price. The true price of cellular broadband is going up fast, worldwide, and like all price rises, it will harm the poorest the most.

Neutral exchange

Paolo Gambini, CMO of Tinet (formerly part of Tiscali), announced that his company has formed an alliance with PCCW of Hong Kong that will give PCCW access to North America and Europe and give Tinet access to Asia-Pacific.

Gambini also said that the company is growing its own EtherCloud platform, which he descrbed as an alternative to ethernet exchanges.

Transatlantic cable

Hibernia Atlantic announced that it will be laying a new transatlantic cable by the middle of next year and offering service on it in Q3 of 2012. The cable aims to reduce latency between the London and New York exchanges by at least 10 percent, to less than 60 ms.

Automated quotes

Greg Hough, CTO of Global Capacity, announced that Lattis Global, its tarriff quoting system, had likely saved 150,000 man hours of work last year by transforming a task that could take hours into one that took several clicks.

The company has added a new offering, One Marketplace Access, whose first customer is MegaPath (which also owns Covad and Speakeasy). OMA delivers specific price quotes for access to specific circuits, making buildouts easier, especially in areas where an ISP does not already have facilities. “We have reduced MegaPath’s SG&A and can help them reach tier 2 and tier 3 markets,” Hough said.

Fiber in New Jersey

Vincenzo Celemente, the very young CEO of Cross River Fiber of Isselin, N.J., announced that it is extending its fiber to the major internet exchange points across the state. The new build will reach important points in the following New Jersey towns: Secaucus, Clifton, Nutley, North Bergen, Newark, Cateret, Edison, Piscataway, Somerset, Rochelle Park, and Totawa.

A great duct system

Hunter Newby, CEO of innovative fiber builder Allied Fiber, praised the contruction of DuPont Fabros‘ Piscataway, N.J. data center as the two companies announced they had connected. Allied Fiber promises to break open its fiber every 3,000 feet as it builds a network across the U.S. (phase one, which is nearing completion, forms a triangle between New York, Chicago, and Ashburn). Allied Fiber found that DuPont Fabros’ data center was only 1,000 feet away from Allied Fiber’s route, which runs along railroad rights of way.

Newby said that the duct system in the Piscataway data center made it very easy to connect to the data center. “The physical infrastructure, multiple points of entry, and the building ducts made this data center unique. As I go across the country, costs vary widely, and costs are mostly determined by landlords, who also vary widely.”

Vinay Nagpal, director of carrier relations for DuPoint Fabros, said that his financial industry customers are using SAN technologies for instant replication of critical data between New Jersey and Ashburn. “Some are able to run SAN applications for up to 180 miles without regeneration.”

Connecting the Toronto exchange

Colleen Gallagher, vice president of business development at First TelecomServices, said that her company has recently lit the last leg of fiber connecting Toronto to its network. The company is a former subsidiary of an energy company and has fiber assets (see map) that connect Chicago, Washington, D.C., and New York and has points of presence in eleven states. “We now offer industry leading low latency connections to Toronto,” she said.

A fiber blog

Metro NS announced that they have launched an ethernet-centric blog.

Every system administrator lacks the time to do everything that needs to be done. “I’m not good at time management,” Limoncelli said, “but I’ve created many coping mechanisms to deal with the problem.”

“It’s not your fault,” he added.

Part of the problem is evolution. The human brain may be suited to surviving in the wild, but it is not suited to time management. For example, one key time management skill is memorizing long lists, a skill not suited to the neocortex, which is roughly the most recently evolved part of the human brain. However, the neocortex is good at making tools. So use them: paper, pencils, smartphones.

Another part of the problem is infrastructure.

If you are your own boss, then you made some tough decisions about infrastructure and you have to compensate for your own lack of investment. Limoncelli remembers a time when he had to persuade companies to adopt trouble ticketing systems.

In his speech, he posted a funny graphic: a trouble ticket that said, “The worst has happened. There’s no coffee!”

Limoncelli added that he has worked at makeshift desks, computer rooms that were too small, and with monitoring systems that could not parse the data they were gathering.

“How many people here have worked at a company that tried to save money by purchasing consumer grade networking equipment instead of professional grade equipment,” he asked. Many in the room raised their hands.

Recommendations

Limoncelli recommended keeping a written To Do list. It can be as simple as a Notepad text document. For Emacs users, Limoncelli recommended Org-Mode, a powerful To Do list system that can be partially automated.

Part of your job is getting rid of interruptions — which is as much about dealing with people as it is about dealing with technology. Limoncelli recommended writing down a request — with large and highly visible hand motions — rather than interrupting what you’re doing and allowing someone else to set your priorities.

Limoncelli recommended what he called “the 4 PM check.” That’s the point in the day when you know whether or not you will be able to do everything that you had planned to do. If you cannot get it all done, he said, you have three options: negotiate an extension, delegate the work, or work late and complete the take.

“Stop working late every night,” he advised.

If you are managing desktops, you should be patching and cleaning machines, backing them up, and making sure that you can restore from backups. Limoncelli referred to the Evard Life Cycle of a Machine.

This is common advice, but Limoncelli had powerful stories about the consequences of not following it. He said that he once came into a company, as a consultant, and found that a key machine was generating a tremendous quantity of security problems. When he walked into the administrator’s office, he did not have to ask why the problems were occurring: he could see, on the desk, in pristine, unopened packages, all of the patches that should have been implemented. The company refused to pay for a test environment for this vital piece of hardware, so the administrator refused to patch it for fear that the patch could take down the machine.

If you have systems, they have to be useful. Limoncelli had several stories about the consequences of burdensome security and backup systems. The result of a burdensome system is that users will avoid it and it will therefore not serve its purpose.

Be a leader

Limoncelli said that administrators must be leaders as well as managers. A manager sets priorities and allocates resources; a leader is the person who goes first and makes it possible for others to follow.

Sometimes leadership means helping others do their job. Limoncelli told the story of a security consultant who was brought in to a company that was growing so fast that machines were being added faster than security issues could be fixed. So the security consultant did what the IT department should have done: she automated many IT tasks so that the IT department could do both tasks: fix security issues and also add new machines.

Limoncelli recommended documenting what you do. He said that the documentation could be bullet points. It could be simple. He recommended documenting the tasks that you least like to do. Eventually, you will be able to delegate them. Also, the documents will allow you to occasionally take a vacation.

Eventually, when your department grows, you can recommend that the company hire a cheaper, junior person who you can mentor. When the company asks you what they will do, you can read from the list of documents you have created. You will be delegating all of the tasks you least like to do.

With good documentation, you may also find that you can automate some of these boring tasks.

People skills

Let’s say you’re approaching your boss. Which of the following do you think would be more effective:

“We need a faster and better server”

or

“I figured out how to reduce the time that our salespeople spend uploading data to the CRM system so that they can spend more time selling.”

If people start e-mailing you instead of using the trouble ticketing system, Limoncelli advised that you be firm but polite. “I always say that it’s nice that they e-mailed me but that I’ll get the message if they use the system and that I’d hate to think that their problem could go unnoticed for a few weeks if I was on vacation.”

Conclusion

Limoncelli is a fantastic speaker. In this schematic summary of his talk, I’ve left out most of the anecdotes that give weight to his recommendations. If you have the chance to attend a speech of his, take it! His tour schedule can be found on his website.

In an earlier interview, Grippi said, “it was the first time it made us think of the violence of those that use your data, and of how, behind the scenes of someone who offers you something for free, there’s always someone that uses the data you exchange with your friends. We deleted ourselves from Facebook and we started to think about an alternative. People don’t really understand the risks they’re taking, but even those that understand them don’t know where else to go to.”

Before starting Diaspora, the club had built a MakerBot and had completed other projects together, such as having the door to the club room tweet every time it was used.

The group decided to build a decent social network for nerds. They went to Kickstarter, whose founder is also a graduate of NYU. Kickstarter allows anyone to raise money for any project through very small donations, as little as $5 per person. The group posted what Grippi called “a pretty terrible video.” The goal was to raise $10,000. In fact, they raised $200,000, which at the time was a record for Kickstarter.

Raphael’s brother Michael was a key part of subsequent development. Michael works at Pivotal Labs, which is now an investor in the Diaspora project and which provides the project its office space in San Francisco (Pivotal also provides breakfast to team members).

Diaspora today

The team has been working on the project for seven months now. Its code base is stored on Github and the project’s public face is the Join Diaspora website.

The project now has 106 contributors. It is AGPL3 compliant.

It is federated. This means that anyone can set up a Diaspora seed (the word “diaspora” is ancient Greek for “scattering of seeds”). Each seed houses the accounts of its subscribers. When people communicate with subscribers of other seeds, the message passes between seeds seamlessly. Currently, the project’s founders operate the largest installation but other large seeds include one in Seattle and another in Germany.

The interface for a user’s profile allows the user to maintain their contact list, aspects, connections, and to connect an account to Twitter and Facebook. The user interface is still in development and is subject to change. An account description is like an e-mail but it is not an e-mail at this time. The structure is username@pod.

Grippi said that where possible, the project tries to avoid reinventing the wheel and instead uses existing protocols, such as Webfinger, hcard, and Salmon.

Sofaer explained that with the hcard protocol, a user can decide whether an image is searchable or not.

For security, the project relies on another trustworthy public protocol: the open SSL RSA library.

The Salmon protocol helps distribute comments. Unlike other social networks, a user cannot see a friend’s comment on a wall if they are not a friend of the person on whose wall the comment was made.

The project currently runs on the Rackspace cloud and stores software images on Amazon S3.

It uses Splunk for log analysis. The group also tracks virtual machine statistics and is especially interested in the time required for specific queries. If a query requires too many milliseconds, there’s an error in the code that must be fixed.

Diaspora in the Future

“Moving forward, we will stablilize and formalize Diaspora,” Sofaer said. He explained that the team’s initial goal was to create software that worked. Now that it’s working, the team needs to formalize how it works in order to ensure that it can scale.

“We wanted to get something up and running without debating protocols for two years,” Sofaer said.

“I want my sister to use it because it’s good for her,” said Grippi. “We want to see many users of free software.”

The eventual goal is that as seeds proliferate, they will compete with each other. Some will offer better privacy, for example.

The team wants to make an API. They want to bring Diaspora to cellular networks with a mobile API. They want to work with existing trust communities, for example by supporting OAUTH.

NYLUG members asked whether Diaspora is ready to participate in the Web of Trust, but the team noted that the technical requirements may be beyond the managers of some seeds. Grippi said that although Diaspora was built for nerds, many people who are not very technical want to participate. He said there are currently about 100,000 members and a waiting list of about 300,000 e-mail addresses.

One part of the task of making Diaspora easy to use is nearly complete. The project has tools called Sod and Chef that are designed to automate the deployment of a seed. Chef is made by OpsCode, and Sod was built by Michael Sofaer.

A user can buy a domain from Rackspace and deploy in three clicks, Sofaer said.

“We’re all developers and no sysadmins, so we needed auto provisioning,” said Sofaer.

Conclusion

The NYLUG group had many tough questions. What happens if someone malicious tries to participate in the project or tries to deploy a bad seed?

But overall, the NYLUG group was very impressed by the project. Attendance was high for NYLUG and the speakers were mobbed after they spoke.

One of Sofaer’s comments stuck with me. He said, “we wanted to make social networking more contextual and to give users more control over what they’re sending and who’s receiving it.”

Before NYLUG, I had attended a speech on the importance of context in the preservation of privacy (see Privacy in Context: A Speech at Fordham Law School).

An earlier version of this story incorrectly said that the Diaspora project team members built Sod and Chef. Thanks to Sofaer for the correction.

Nissenbaum said that she thought this would be the easiest book she’s ever written, because it was simply synthesizing many papers, but that in fact it was the hardest, and it took two years. In the book (and in her current work) she is building an analytical framework that would identify the aspect of an online transaction or interaction that causes social anxiety.

The miracle of the internet is about the rapid dissemination of information. This has delivered powerful economic benefits, and it has delivered freedom.

The internet has also enabled massive data repositories that have caused concern. Nissenbaum mentioned Choicepoint and you can see the concerns of the Electronic Privacy Information Center here. Nissenbaum also mentioned the Total Information Awareness (TIA) program of the federal government. Of TIA, EPIC wrote, “TSA has failed to meet its legal obligations for openness and transparency under the Freedom of Information Act and has violated the spirit of the Privacy Act for the protection of privacy rights.”

Getting the terms right

Nissenbaum said that when we talk about privacy, we need to reject the dichotomy between public and private. If all information is available on the internet, what matters is not whether it’s there, but the context in which it was posted to the internet.

Her prescriptions can be broad. In 2004, she wrote a paper on surveillance called Privacy as Contextual Integrity in which she argued that surveillance footage, for example, must be used within its security context but must not be exported outside it.

She wrote, “when violations of norms are widespread and systematic as in public surveillance, when strong incentives of self-interest are behind these violations, when the parties involved are of radically unequal power and wealth, then the violations take on political significance and call for political response.”

In her speech, Nissenbaum said that information transmission is critical to society. In the paper, she noted that many surveillance activities today could violate any law that attempted to codify our social norms and expectations of privacy. In her talk, she noted that any overly broad law would have a serious negative effect on society and on the benefits that the internet has brought to the economies and societies of the world.

In a conversation, she noted, we control the transmission of information. We talk to a specific person. In the context, we understand how our friend will use the information. In another context, such as in a therapy session, one person talks and the other listens and the conversation remains confidential.

“Privacy issues arise when everyone instead of a specific person receives the information,” Nissenbaum said.

Nissenbaum said that laws already understand that privacy issues invole one actor transmitting information and another receiving and then using that information. See, for example, section 6802 of the Gramm-Leach-Bliley act which begins:

Except as otherwise provided in this subchapter, a financial institution may not, directly or through any affiliate, disclose to a nonaffiliated third party any nonpublic personal information, unless such financial institution provides or has provided to the consumer

The act recognizes the obligations of the recipient of private information (the financial institution) with regard to information received from an actor (the consumer).

The internet brings change

“The world is changing, upsetting entrenched norms,” Nissenbaum said. She said that the changes affect vested interests but they also effect core values. In order to evaluate privacy questions, we must consider the context.

For example, she noted that the results of HIV tests must remain private or those who are infected or might be infected will avoid health care.

She said that PACER has put all federal cases online but added that there is debate — good debate — occurring at the state level.

She said that data mining can be good. It can improve health care and education. But many people are not aware of the scale and scope of current data mining by businesses. She recommended a series of articles in the Wall Street Journal called What They Know About You. Of particular interest to readers: After sharp internal debate, Microsoft designed Internet Explorer so that users must turn on privacy settings every time IE starts.

If human beings become concerned about the privacy implications of online activity, Nissenbaum noted, they might stop shopping online, which would be bad.

A Fordham student noted that although she herself has opted out of Facebook, she still worries that her friends will post photos of her online.

The systems that Nissenbaum is developing that will be used to analyze the context of online transactions seem most useful for online venues where structured activity takes place. For example, as states analyze the privacy implications of online educational record systems, her schema will provide a valuable framework.

But in areas of the internet that more closely resemble an actual conversation, it seems unlikely that any schema can list all of the layers of meaning and context that surround an actual conversation between friends.

Nissenbaum’s talk shows that while governments can solve many of their privacy issues, social networks face greater challenges. This was timely: after visiting Fordham Law School, I went to a meeting of the New York Linux User’s Group (NYLUG) where the open source Diaspora social networking project would present (see Diaspora at NYLUG).

And so journalism declined.

The Federal Trade Commission, worried about the decline, considered trying to stop it by subsidizing newspapers, a move that many on both the left and right said confused “newspapers with journalism.”

In fact, there are journalists, but they no longer work for newspapers, and news is no longer broken by newspapers. News breaks on the internet.

At a recent symposium that I covered here, Harvard Law School professor Yochai Benkler said that the broadcasters’ and newspapers’ eagerness to ask the Obama adminstration for permission to publish the Wikileaks story means that they will never break important news in the future. “The next Ellsberg will not risk his career and liberty to go to The New York Times. The whistleblower will publish on the internet and the media will report on it.”

Leaks are inevitable in the future, as The Economist recently noted. These leaks will no longer be screened by the editorial function of newspapers but will instead be dumped. The raw data will be available to everyone on the internet.

Some are arguing for a fascist response, and others hope that future leaks harm the powers that be. As the Economist article cited above noted, “Some of us wish to encourage in individuals the sense of justice which would embolden them to challenge the institutions that control our fate by bringing their secrets to light. Some of us wish to encourage in individuals ever greater fealty and submission to corporations and the state in order to protect the privileges and prerogatives of the powerful, lest their erosion threaten what David Brooks calls ‘the fragile community’ — our current, comfortable dispensation.”

In a separate article, The Economist argued that the U.S. government should accept current and future wikileaks, arguing for prosecution but not for persecution:

Calling Mr Assange a terrorist, for example, is deeply counterproductive. His cyber-troops do not fly planes into buildings, throw acid at schoolgirls or murder apostates. Indeed, the few genuine similarities between WikiLeaks and the Taliban—its elusiveness and its wide base of support—argue against ill-judged attacks that merely broaden that support. After a week of clumsy American-inspired attempts to shut WikiLeaks down, it is now hosted on more than 700 servers around the world.

The big danger is that America is provoked into bending or breaking its own rules, straining alliances, eroding credibility and — because it will not be able to muzzle WikiLeaks — ultimately seeming impotent. In recent years America has promoted the internet as a menace to foreign censorship. That sounds tinny now. So did its joy of hosting next year’s World Press Freedom Day this week. Chinese and Russian glee at American discomfort are a sure sign of such missteps.

The best lessons to bear in mind are those learned in such costly fashion during the past decade of the “war on terror”. Deal with the source of the problem, not just its symptoms. Keep the moral high ground. And pick fights you can win.

One thing is certain: the newspapers will no longer insulate governments and corporations from the consequences of the revelation of secrets.

Thanks to Joly MacFie for the video.

Text of the speech:

Today, on OneWebDay, we want to urge the FCC to assert its right to protect the three key principles of the internet, principles that have made it friendly to innovation and competition. We want the FCC to insist that the internet be open, that computers be connectible from end to end without interference, and that internet management be open and transparent — not secret and in the service of the companies who pay for it to be the way it is.

OneWebDay was founded in 2006 to celebrate the internet, a uniquely organized piece of infrastructure that has become critical to all of our lives, directly for those who use it, and indirectly for those who use the services that now depend on it, such as education and banking and healthcare.

When I first read about the internet, in Ed Kroll’s book, the Whole Internet User’s Guide and Catalog, published in 1993, it was described as a cloud. The reason it was described this way is that the internet was not built from the top down. It was built by network administrators at military institutions and at universities, who built local networks and allowed those local networks to connect to each other.

This is the first principle of the internet, the End to End principle. It says that any machine must be able to connect to any other machine, and that any machine can use any application. There can be no internet gatekeeper to decide what can and cannot be done.

Not everyone likes this. The internet receives many of the criticisms that are leveled at democracy. The internet is less efficient than it could be, if it had managers. The internet is less secure than it could be, if we could police it by reading everyone’s mail. The internet will be brought down by the weight of the people that make it up, because they will inevitably act against the interests of the internet, and you only have to destroy democracy once for it to never return.

The internet when it began was an American thing. It was built by our military and the educational institutions that work with it. It was built on American principles and one of those is openness. Another is that the whole world wins when science and technology advance.

The internet was built on the principle of openness. The second principle is that users should be allowed to know how the internet works. Many technologies are secret. There are parts of your cell phone, if you have one, that you’re not allowed to change. But the internet is open, and that means that it can be changed not just by an institution with a big research budget, but also by two guys like Sergey Brin and Larry Page (and some California venture capital). It can be changed by the guy who built Facebook who, if the latest Hollywood movie is at all accurate, may not be a nice guy.

It can be changed by you and me. Any of us can build a web page, start a blog, and participate in an internet community. It can be changed by a speech at NYU by Professor Eben Moglen who inspired the Diaspora open source challenge to Facebook with a speech that included the phrase, “I’m not suggesting [Facebook] should be illegal. It should be obsolete. We’re technologists, we should fix it.”

Not everyone agrees that the internet should be this way. Some might want users to take a test to qualify to build a web page. In South Korea, they want a crackdown on celebrity gossip and internet suicides. Here in the U.S., a recent comment post to the FCC blog says that the internet should be illegal because it contains pornography, and many around the world might agree with that. In some parts of the world, they want to make defaming a specific religion illegal.

As long as there is free speech in a democracy, there will be speech that is abhorrent to some people. But if you forbid that free speech, that you shut down a key component of democracy.

Perhaps the most startling aspect of the internet model is the cloud itself. In reality, there is some management within the cloud, such as DNS servers and internet registrars, and ICANN itself, on whose board Susan Crawford once served.

But although the internet is managed, it is owned and maintained by its constituents. It is built out by internet service providers large and small, and run by an international community of network managers who can, when the need arises, fight problems such as organized crime together but who for the most part spend their days making sure that their piece of it runs well.

Sharing, openness, and local ownership are not really big on the agenda of most companies and governments today. But they’re what makes the internet work and what keeps it unique. We’re here today to say we want to keep it this way.

Cases:

story:
http://www.dslreports.com/shownews/60996

FCC:
http://hraunfoss.fcc.gov/edocs_public/attachmatch/DA-05-543A1.pdf

FCC Identifies, Fines VoIP Blocker
Madison River Communications pays $15K
(old news – 12:40PM Friday Mar 04 2005)

These issues have not just cropped up this year. They have been with us for a long time. In the past five years, however, there has been one big change. Until 2005, the internet mostly served to disintermediate non-telecoms businesses, such as bookstores, the postal service, and other intermediaries who worked between businesses (B2B) or who connected businesses to consumers (B2C).

Shortly before 2005, VoIP went mainstream. It had been around for a long time. Jeff Pulver had built his VoIP-centric career on the technology starting in 1999 and even earlier. In 2005, a local phone company called Madison River Communications was fined by the FCC for trying to keep Vonage phone calls off its network.

(There are persuasive conspiracy theories, such as Fred Goldstein’s, about the fine.)

What concerns us here is that the FCC could have asserted a general principle. It could have said that the internet must remain open, that any machine on the internet must be able to connect to any other machine on the internet, and that management of the internet must be open and collaborative.

The FCC made no such assertion.

In the consent decree , the FCC said it investigated in order to allow customers their choice of VoIP provider: “the Bureau inquired about allegations that Madison River was blocking ports used for VoIP applications, thereby affecting customers’ ability to use VoIP through one or more VoIP service providers.”

Madison River settled, for a mere $15,0000, and the FCC was therefore able to avoid spending money investigating Madison River and Madison River avoided potentially high legal expenses: “To avoid the expenditure of additional resources that would be required to further litigate the issues raised in the Investigation, and in consideration for the termination of the Investigation in accordance with the terms of this Consent Decree, Madison River agrees to make a voluntary payment to the United States Treasury, without further protest or recourse to a trial de novo, in the amount of fifteen thousand dollars ($15,000.00) within ten (10) business days after the Effective Date of the Adopting Order”.

Madison River promised not to do it again: “In order to resolve and terminate the Investigation, the Bureau requires, and Madison River agrees, that Madison River shall not block ports used for VoIP applications or otherwise prevent customers from using VoIP applications.”

The FCC could have asserted its right to regulate the three core internet principles. It could have argued that its role is to protect the consumer and the internet. Instead, it argued the narrowest of principles and achieved the narrowest of goals. It asserted the right to police competition among VoIP providers on privately-owned networks.

FCC vs. Comcast

This is the most important net neutrality case in the USA to this day because Comcast fought the FCC and won, fundamentally challenging the FCC’s power to regulate the internet.

DSL Reports reported in August of 2007 that Comcast was using an internet traffic management device from Waterloo, Ontario-based Sandvine to throttle BitTorrent users who were sending more traffic outside of Comcast’s network than they were bringing in. The Sandvine device specifically targeted sharers and did not target downloaders, according to DSL Reports forum members:

“The Sandvine application reads packets that are traversing the network boundary. If the application senses that outbound P2P traffic is higher than a threshold determined by Comcast, Sandvine begins to interrupt P2P protocol sequences that would initiate a new transfer from within the Comcast network to a peer outside of the Comcast network. The interruption is accomplished by sending a perfectly forged TCP packet (correct peer, port, and sequence numbering) with the RST (reset) flag set. This packet is obeyed by the network stack or operating system which drops the connection.”

For internet users, the key phrase here is “perfectly forged TCP packet” because it meant that Comcast was intervening in their use of the internet without telling users. It was choosing which machines users could connect to. It was not being open and transparent in its network management.

In 2008, the FCC said that this behavior was illegal, and the Electronic Frontier Foundation (EFF) released a neutral tool called Switzerland to help users determine whether their own ISP was doing the same.

Note that the Sandvine device had been on the market for some time. Equipment makers — not just Sandvine — had been developing products that could do this. They had been marketing them openly. Had the FCC been concerned about fundamental internet principles it (or Congress) could have intervened. They chose not to.

In 2010, the Comcast won a lawsuit against the FCC in which Comcast argued that the FCC had overstepped its authority.

This was huge news. It threatened the National Broadband Plan. It fundamentally challenged the FCC’s power to regulate the internet. The challenges raised by this case have not been resolved by the FCC, although Google and Verizon are eager to solve them for the FCC, which is an issue we’ll discuss next.

The New York Times calls current FCC policy “the struggle for what we already have.”

“Since that ruling came down in March, the agency has been going down two tracks at the same time. It has been desperately trying to find a way to re-establish jurisdiction over broadband services, while at the same time continuing to push for net neutrality. It has become a very complicated dance.”

The Times thinks that “consumers have come to expect an open Internet, and companies will violate net neutrality at their peril. That is just the way the Internet has evolved,” but such a statement ignores the growing market power of the largest ISPs.

Verizon is a major cell phone company as Verizon Wireless. It provides regular telephone service as Verizon. With Verizon FiOS, it is the largest fiber optic provider in the U.S. and offers television service on its fiber lines.

Comcast is a major cable company. It also is one of the nation’s largest telephone companies with its Digital Phone offering. It competes with Verizon to be the largest ISP in the U.S. Comcast wants to buy NBC.

Commissioner Copps noted in a Washington Post editorial that the FCC could retain authority to regulate the internet if it asserted that the internet is “Title II” (common carrier telecommunications) rather than “Title I”.

He wrote:

This was a predictable outcome of FCC actions during the Bush administration that consciously moved broadband Internet access from Title II, which would have supported the commission’s authority, to a murky place that invited court challenge.

This was a major flip-flop from the historic — and successful — approach of forbidding discrimination on our communications networks. Now is the time to put broadband back under Title II, where it belongs — and under which many smaller companies continue to offer Internet access to the public.

Nor is this debate about regulating the Internet. It’s about whether consumers or a few huge Internet service providers will control consumers’ online experiences.

Today, on OneWebDay, we want to urge the FCC to assert its right to protect the three key principles of the internet, principles that have made it friendly to innovation and competition. We want the FCC to insist that the internet be open, that computers be connectible from end to end without interference, and that internet management be open and transparent — not secret and in the service of the companies who pay for it to be the way it is.

Case: Google and Verizon

In August of this year, Google and Verizon posted a joint proposal for the regulation of the internet.

There were many great responses, but I want to highlight that of Susan Crawford, the founder of OneWebDay. She called her post “Leadership,” highlighting a vacuum at the FCC. She wrote:

I don’t blame them for trying. These are profit-making companies, not regulators and not advocacy groups. I’m sure Google, in particular, is tired of the multi-year slog that has gone into the net neutrality debate – and tired of being made the poster-child for the issue as other major online companies have steadily backed away.

The key takeaway from today’s announcement is that it underscores the urgency of FCC action to ensure that it has jurisdiction to speak to American companies about high-speed Internet access. As someone told me today – snappy line – the agency is being disintermediated.

Funny, but not good for the American economy. These two giants have found a policy vacuum and they have, understandably, filled it. A private deal – even a private deal aimed at suggesting legislation – is not the same thing as acting in the public interest. That’s the FCC’s charge.

The key tradeoff being made here is between the treatment of wireless services, on the one hand, and the treatment of nondiscrimination, on the other. Google gave on wireless, and so there’s no policy suggestion for wireless net neutrality that has been provided by the companies. That’s a huge hole, given the growing popularity of wireless services and the recent suggestion by the Commission that we may not have a competitive wireless marketplace. Verizon gave on nondiscrimination, and so there is a suggestion that paid prioritization of services over the Internet would be presumed unlawful (something that AT&T would not have agreed to).

Both companies left “managed services” (or “other services”) off the table for regulation. That’s a giant, enormous, science-fiction-quality loophole. It means that Verizon could decide what bits reach consumers more quickly; it means they’ll be able to favor particular uses of Internet access for exclusive deals. It’s the exception that swallows the rule, as lawyers like to say. It’s prioritization using another label. There’s a save in there that suggests that the “other service” has to be distinct in scope and purpose from Internet access (something cable would not have agreed to), but that’s a long way from an enforceable standard.

What’s needed now is leadership. Here’s a quote sent to me yesterday – from LBJ to regulators: “Let the venal and the self-seeking and the tawdry and the tainted fear to enter your building.” You need backbone to regulate a giant industry, and this one is too important to our economic, social, and cultural future to ignore.

Susan Crawford is devoting most of her blog time (and, I assume, much of her real world time) to opposing the merger between NBC and Comcast, an issue we can discuss if you’d like to.

Case: AT&T Censors Pearl Jam’s Anti-Bush Lyrics During Live Webcast

http://gizmodo.com/287775/att-censors-pearl-jams-anti+bush-lyrics-during-live-webcast

http://arstechnica.com/old/content/2007/08/pearl-jam-censored-by-att-calls-for-a-neutral-net.ars

How bad could it be? Net neutrality advocates’ worst fears were realized in late 2007 when AT&T censored anti-Bush lyrics in a performance by Pearl Jam. When the band transitioned one of its songs into “the melody from Pink Floyd’s ‘The Wall,’ … Eddie Vedder served up a pair of anti-Bush lyrics to the tune. ‘George Bush, leave this world alone,’ he sang. ‘George Bush, find yourself another home.’ ” People at the concert heard the lyrics. People listening in online heard only silence.

The band called for net neutrality. “What happened to us this weekend was a wake-up call, and it’s about something much bigger than the censorship of a rock band.”

As far as I know, the FCC did not interfere or even investigate. Any regulator concerned about core principals would have at least looked into this. FCC Commissioner Copps did state, “nobody should have that power to do that and then be able to exercise … control over the distribution and control over the content too.”

AT&T weaseled out by blaming a contractor that it hired to produce the streaming video.

Censorship around the world

We in the U.S. set the standards for freedom from censorship and right now we’re not doing very well. The internet around the world is becoming more filtered. Nation states are trying to build walls to keep parts of the internet out, and it’s not just repressive regimes.

The United States itself, under George Bush, created the Total Information Awareness system that may have recorded every internet voice call and e-mail sent in or through the USA.

As recently September 10th, the ACLU called on President Obama to disavow the program. He has not done so.

Meanwhile, Google warns that almost every nation is asking that content be taken down. Here is a list of takedown requests. Note that the majority of the takedown requests come not from dictatorships but from the democracies.

Australia plans to go ahead with a futile but intrusive filter. Australia’s communications minister Stephen Conroy says that the filter will protect citizens from child porn. Australia’s opposition says it will only give parents a false sense of security and will slow down the internet.

Perhaps the most famous internet censorship operation is China’s. Google was nearly kicked out of the country but has been allowed to stay for another year after redirecting its China website to Hong Kong. A one-click redirect displeased Chinese authorities, but a redirect requiring two clicks has satisfied them. The censorship involves the government and the nation’s ISPs, according to Human Rights Watch.

Pakistan has blocked YouTube and Facebook over what it calls “sacrilegious images”.

Censorship of the Blackberry:

Nations as diverse as the United Arab Emirates and India have asked Blackberry to set up a server in their country so that all messages could be read by the government. “The BlackBerry, then, offers a way to get information to a server outside a country without having anyone inside that country read it. The key here is the location of the server; a country is generally happier when it has all servers in its own warm jurisdictional embrace,” noted The Economist.

Case: Comcast – NBC Merger

http://scrawford.net/blog/comcast-nbcu-forum-today-in-chicago/1372/

http://scrawford.net/blog/sports-just-the-most-obvious-indicator/1392/

The history of internet regulation is a history of regulators trying — and failing — to catch up with what’s actually happening. It’s like the Sandvine box that was in development for years before the FCC noticed the results and tried to stop it. Susan Crawford appears to think that the most important challenge to the internet today is epitomized by the potential merger of Comcast and NBC.

She argued: Comcast’s market power in distribution currently gives Comcast control over programmer behavior and will allow it to protect the leading sports, business, and entertainment brands it gains from this transaction

Comcast’s current relationships with programmers, enhanced by its pricing and bundling control over key NBCU cable channels and online properties, may allow it to raise the costs of its online and offline distribution rivals

High-speed Internet access prices paid by Americans may get higher

But most importantly: Experience with prior mergers for which FCC exacted conditions demonstrates that DOJ should be involved in enforcement. Examples include failures to enforce fiber commitments in SBC-Pacific Telesis, SBC-SNET, and NYNEX-Bell Atlantic; competition commitments in SBC-Ameritech, Bell Atlantic-GTE (Verizon; and promised cost savings in AT&T-BellSouth.

Here’s a detailed example from Crawford: “Comcast has 61% of the Chicago market; 63% of Philadelphia; 58% of San Francisco; 59% of Miami. This power will be increased by the addition of NBCU content – particularly sports content.”

Many argue that monopoly power and pricing power can be exercised by any company that has at least 40 percent of market. As you can see, Comcast is far beyond that, especially in its home base of Philadelphia.

Conclusion

Defending these abstract principles: openness, end to end architecture, and transparent management, requires that we watch the marketplace with vigilance, as Susan Crawford is doing. Today, on OneWebDay, we celebrate the internet for what it is. This is the age of the internet. But the internet may change, and not for the better. Powerful companies want to control it. Stay informed and stay active.

The key difference between then and now is that through 2002, the ISPs still had control of the market, but today, the phone and cable companies rule.

Here are snapshots of the market from Q4 2002 and Q3 2008.

In 2002, the top 5 ISPs were AOL, MSN, EarthLink, United Online, and Comcast. Together, the top 5 ISPs controlled just over 50 percent of the market, which was already too much.

In 2008, the top 5 ISPs were SBC, Verizon, RoadRunner, Verizon, and AOL (representing 4 companies). Together, these four companies controlled about 56.2 percent of the market.

In 2002, I estimated that independent ISPs had about one third of the market and comprised three of the top 5 U.S. ISPs.

In 2008, I estimated that independent ISPs had none of the top spots and that EarthLink, ranked sixth, was doomed. I estimated that independent ISPs had much less than one quarter of the U.S. market.

Today, independent ISPs have an even smaller share of the market.

What the new administration is doing right

The Obama administration aims, through the stimulus, to fundamentally change the ISP business. The ideal depicts a local business serving a rural community and helping it thrive in the face of negligence from the large monopoly internet providers.

The data that the FCC is using ignores the fact that the phone companies are ditching landlines by selling them to entities that will surely go bankrupt.

AT&T has already filed for permission to shut down its copper network. AT&T believes that the future is VoIP, which explains why AT&T has tried to prevent a free market in VoIP services.

What the FCC is doing wrong

Small businesses are harmed more by regulatory uncertainty than by any actual action of the FCC. In an environment in which investors and entrepreneurs do not know whether they will have permission to run their businesses, conspiracy theories are running rife.

The FCC appears to be embroiled in chaos of the worst kind. It is preparing to try to enact rules that were just ruled illegal by the DC Circuit Court.

The FCC should take this an opportunity to change the way the internet is regulated. The internet should be regulated as one entity — as today’s Waxman Rockefeller letter (.pdf) so rightly notes. The false splitting of the internet baby into an information service and a telecommunications component dates back to a disastrous anti-competitive finding from 2002.

A bold FCC would articulate a vision for how the internet should be regulated and then proceed to figure out how to get there. A bold FCC would define the internet as a utility over which the services of the 21st century flow in an unprecedented, connected manner.

One such vision was articulated in 2002: take away the monopoly protections and let the legacy networks fail fast. Current policy is to allow these legacy networks to lie and to pile debt on the backs of companies that will soon fail. Current policy is the opposite of fail fast — it is a constant bailout in which ever more government cash is devoted to businesses that could not survive a genuinely competitive and free market.

The New York Times called for clear regulation of the internet (Verizon opposes the idea). The most popular comment on the editorial said, “The giants controlling the market have no interest in ensuring that everyone has access at all, much less equal access. And they certainly have no interest in competitive pricing.”

I agree.

The ISP business just needs honesty and sanity

The ISP business should be a reliable and dependable business. Customers pay the same fee every month (or pay up front for six months to a year) and the ISP delivers the same reliable service to customers day after day.

Uncertainty in the ISP business comes not from customer behavior — ISPs with a sufficient number of customers can generally predict customer behavior in a manner similar to that employed by insurance company actuaries. Instead, uncertainty comes from the behavior of the monopolies. Every price change affects the industry.

All too often, independent ISPs advertise the actual price of service while the monopolies do not. Cellular pricing is even more opaque than wireline, though a few are working hard to decode prices every time they change.

The FCC has never enforced truth in billing rules.

The recent national broadband plan made some positive recommendations for forcing the monopolies to stop lying about broadband speeds, but failed to do anything about pricing lies. Doing nothing about the problem is not a plan. The rip offs continue.

This distorts the market. Customers have to compare the real prices posted by independents with fake heavily advertised price points set by the monopolies. Few are able to do so.

While much FCC policy has been inward looking, refusing to treat the world as a laboratory in which alternate polcies are tested, some failing and some succeeding. Both failures and successes provide useful lessons.

Two representatives of successes were present, Herman Wagter of citynet.nl in Amsterdam and Johan Henæs Norwegian equipment maker INS Communications.

Lessons from Europe

Wagter pointed out that up to 90 percent of the U.S. populations lives in areas that are at least as densely populated as Europe, and therefore that the same economics of density that favors high speed deployments there should favor high speed deployments in the urban and suburban areas of the U.S.

Wagter also pointed out that middle mile deployments can be repurposed to other customer while last mile deployments can serve only the one home to which they connect. This makes middle mile projects safer to build, in theory.

Finally, Wagter said that fiber would be a good stimulus project because it delivers jobs. “10 percent of the cost of a fiber project is the fiber, 10 percent is the equipment, and 80 percent is the labor (although labor costs decrease in rural areas).”

In his experience, it costs about 900 Euros to connect each home, far lower than the $3,000 estimate common in the U.S.

Henæs said that a lesson learned in Europe is that nobody can predict what broadband will be used for. The fiber builders expected to deliver a specific set of services but could not imagine apps like YouTube.

He agreed with Wagter that fiber builds should be open. “Fiber builds should not only support FTTH,” he said. They should also support wireless backhaul, business-to-business service, and enable the consolidation of COs. I would add that fixed wireless, not just cellular, should be part of any fiber middle mile project.

Benoit Felten, Yanke Group’s fiber analyst, contributed to the global perspective by showing the panel where 100 Mbps service is available — and the three places in the world where 1 Gbps service is available (Japan, South Korea, and Hong Kong). He predicted that in the future, 1 Gbps service will be available across the world.

Of course 100 Mbps service is available in the U.S., from Cablevision, but it is asymmetrical and not widely available — and it costs more than the same service anywhere else in the world.

Lessons from the U.S.

Tim Nulty, project director of the Eastern Vermont Community Fiber Network said that fiber could be deployed nationwide across the U.S. now.

He pointed out that the Universal Service Fund (USF) spends about $7 billion each year — equivalent to the broadband stimulus. “With it, we could wire the entire rural America, 40 million people,” he said (I think he was excluding Alaska).

He said (and Felten backs me up on this extraordinary quote) that anywhere where you can get 10 subscribers per mile, you can build fiber. At 20 people per mile, each fiber node supports 6,000 to 8,000 subscribers. At 10 per mile, each node supports about 30,000 pepople.

He added that if you build fiber properly, the network is transparent to the technology you hang on it. He said that EC Fiber is able to upgrade.

Joanne Hovis, of NATOA, said that there are 57 municipal FTTP networks in the U.S., mostly in rural areas, plus countless county networks that serve the government and educational and utility providers.

Craig Settles tweeted a link to 10 profiles of community fiber networks.

The lesson from the U.S. is this: it is possible to build fiber anywhere (except maybe Alaska).

John Cioffi, the DSL professor, noted that many fiber deployments use DSL for the last mile, and that fiber deployments are small compared to the number of customers served by DSL.

David Reed of MIT said that Wi-Fi MIMO (802.11n) is far superior to 4G cellular. I agree that for the past decade, cellular interests have unfairly presented themselves as the sole providers of wireless broadband, when cheaper, easier to deploy systems were available and are still available — and the cheaper fixed wireless systems are also better.

“Wireless is much more than just a third pipe,” said Reed.

Reed also made a complex point about personal mobility. He said that he believes the application of the future will be the “amulet” by which he meant the applications that use your identity — your contacts, you location, you activities.

Fiber’s business model should be the support of local wireless networks, according to Reed.

More recently, Brodsky claimed that bids were rigged in Connected Nation’s favor in the state of Florida, arguing that there was no other explanation why the highest bidder won a broadband mapping contract.

Maps are important. They show where the government should invest money. They say who has broadband and who does not. If the maps are drawn by the phone companies, they could direct stimulus money only to the areas they don’t care about, bypassing wealthy areas they would like to deliver service to but have not yet built out.

Today, NYPIRG is calling out such policies. In its report (available in .pdf format here), NYPIRG says, “Contracts or grants to map data … must include requirements
that the mapping entity disclose any financial or other relationships to broadband providers. If data are self reported by a broadband provider and not independently verified, that should be disclosed and the data should not be considered accurate until independently verified.”

The report does not specifically name Connected Nation, but readers understand that’s the problem that’s addressed by this recommendation — a recommendation that is so obvious that it should not have to be said.

The report contains a massive number of other good ideas, endorsing structural separation, better data collection, an internet literacy curriculum and more.

NYPIRG’s report is a masterpiece.