Is Carnivore All-Consuming?

20 November 2000

Date: Sun, 19 Nov 2000 18:21:40 +0000 (GMT)
From: Jim Dixon <jdd@vbc.net>
To: George@orwellian.org
cc: cypherpunks@cyberpass.net
Subject: Re: Carnivore All-Consuming

On Sat, 18 Nov 2000 George@orwellian.org wrote:

> EPIC FOIA...
>
> http://www.latimes.com/wires/20001117/tCB00V0387.html
>
> WASHINGTON--The FBI's controversial e-mail surveillance tool,
> known as Carnivore, can retrieve all communications that go
> through an Internet service -far more than FBI officials have
> said it does -a recent test of its potential sweep found,
> according to bureau documents
> [snip]

Carnivore is an NT-based PC. How could it conceivably process all communications through even a mid-sized ISP?

There are at least two problems: processing power and network architecture.

As regards the first, our customers, many of them smaller ISPs, find it necessary to employ NT clusters to handle subsets of their traffic (Usenet news, Web proxies, and so forth). So it is difficult to believe that a single NT box could monitor their entire traffic load.

As regards the second, most ISPs of any size have multiple PoPs and multiple high-speed connections to other networks. It would require incredible contortions to route all of their traffic to one point for monitoring. And for the larger network, the bandwidth into that single point would be unmanageable.

The UK government proposed building something more sophisticated than Carnivore. Consultants led them to believe that this was feasible, and costed a solution. The UK ISP associations (the LINX and ISPA) replied to their proposals by saying that (a) the proposals showed no understanding of the technical structure of the Internet and (b) their cost estimates were ridiculously low, even if the Internet could be distorted sufficiently to be monitored in the manner envisioned.

As far as we can see, the UK government as an institution is not capable of even understanding the Internet. They simply do not have enough competent technical staff. They do have a lot of relatively senior people who claim to be competent - and give bad advice, some of which finds its way into legislation and programs of action.

The overall capacity and the complexity of the Internet is increasing at an explosive rate. For better or for worse, this far exceeds the growth in any government's capability of monitoring Internet traffic.

Jim Dixon VBCnet GB Ltd http://www.vbc.net
tel +44 117 929 1316 fax +44 117 927 2015

Date: Sun, 19 Nov 2000 18:13:20 -0800
To: Jim Dixon <jdd@vbc.net>
From: Steve Schear <schear@lvcm.com>
Subject: Re: Carnivore All-Consuming
Cc: cypherpunks@cyberpass.net

At 06:21 PM 11/19/00 +0000, Jim Dixon wrote:

[Snip message above]

A PC, using off-the-shelf HW, is capable of filtering a full 100 Mbps link (144K packets/sec) as demonstrated by the BlackICE products http://www.networkice.com/html/blackice_sentry.html

steve

Date: Mon, 20 Nov 2000 03:20:35 +0000 (GMT)
From: Jim Dixon <jdd@vbc.net>
To: Steve Schear <schear@lvcm.com>
cc: cypherpunks@cyberpass.net
Subject: Re: Carnivore All-Consuming

On Sun, 19 Nov 2000, Steve Schear wrote:

[Snip message above.]

First, like any other manufacturer's claims, these should be treated with some skepticism.

Second, this is an intrusion detection system. I suspect that they are looking for something simpler than what Carnivore is trying to detect.

Third, even if you believe that they can really analyse data at 100 Mbps, this still doesn't give them the ability to handle more than one PoP with two DS3 connections. This is still orders of magnitude away from being able to handle a major site with multiple 2.5G connections, let alone all of the traffic handled by a major ISP.

The original claim was that Carnivore could monitor all of an ISP's traffic. This isn't true for most ISPs. And the amazing growth rates that we are seeing in bandwidth and network complexity make it exceedingly unlikely that Carnivore or anything like it will ever catch up.

Qwest deployed 14,000 miles of fibre some years ago. This was packaged as conduits carrying 48 fiber pairs, each pair using wave division multiplexing to carry 8 to 16 optical channels, with each channel running at 10 Gbps. That's 160 Gbps per fiber, 7,680 Gbps per conduit. Qwest is one of many carriers. 160 Gbps over a fiber pair isn't state of the art. Qwest has many conduits.

If a PC can monitor 100M of bandwidth, it would take, uhm, about seventy seven thousand PCs to monitor one of Qwest's conduits. Not that I believe that one PC can monitor traffic at 100 Mbps.

> >The overall capacity and the complexity of the Internet is increasing
> >at an explosive rate. For better or for worse, this far exceeds the
> >growth in any government's capability of monitoring Internet traffic.

Jim Dixon VBCnet GB Ltd http://www.vbc.net
tel +44 117 929 1316 fax +44 117 927 2015

Date: Sun, 19 Nov 2000 21:37:11 -0800
To: cypherpunks@cyberpass.net
From: Bill Stewart <bill.stewart@pobox.com>
Subject: Re: Carnivore All-Consuming

On Sun, 19 Nov 2000, Steve Schear wrote: [Snip.]

At 03:20 AM 11/20/00 +0000, Jim Dixon wrote: [Snip.]

Actually, "most" ISPs probably don't have more than two T3s or OC3s, because most ISPs are the 5000+ little ones; many only have a few T1s. But big ISPs are a different issue; any of the Tier 1 providers could melt a Pentium box if they directed a moderate fraction of their traffic at it.

The question is how the carnivores tell the ISP's network what they're looking for, and how much cooperation they need from the ISP. Most ISP traffic is probably web, not email, and the email that's actually handled by ISPs (as opposed to just passing through) is handled by big mail servers that could perhaps be told to forward all mail for targeted accounts, since they need to do that level of indentification to handle the mail in the first place.

For email, the big player is of course AOL, followed by specialized mail providers like iname.com, and the portal sites like Excite, Yahoo, and Hotmail, and a few ISPs like Earthlink/Mindspring. (The business has gotten sufficiently specialized that I'm not sure how many of those sites really provide their own service rather than outsourcing to specialists.) As with big ISPs, if they cooperate, the job's possible, and if they don't it's pretty intractable.

If you know your target's IP address, it's a lot simpler - get the routing protocols to shove their traffic your way by advertising routes using OSPF, BGP, or whatever.

>Qwest deployed 14,000 miles of fibre some years ago. This was
>packaged as conduits carrying 48 fiber pairs, each pair using
>wave division multiplexing to carry 8 to 16 optical channels, with
>each channel running at 10 Gbps. That's 160 Gbps per fiber,
>7,680 Gbps per conduit. Qwest is one of many carriers. 160 Gbps
>over a fiber pair isn't state of the art. Qwest has many conduits.

They do have a nice _little_ network :-) Actually, most of that fiber isn't even lit yet, much less full, and much of their bandwidth isn't ISP traffic, it's private line sold to businesses or other ISPs. The last AT&T marketing hype I saw placed us as #2, well behind UUNET. The real bandwidth constraints are mainly the routers - most big ISPs use Cisco 12000 GSRs or products from Juniper or other emerging competitors, most of which like to call their products "terabit" routers because they have reasonably large backplane capacity.

A totally different bandwidth segment is inside the big hosting centers - Exodus, Globalcenter, etc. Most of that's Gigabit Ether, with various brands of switches and routers, and an amazing fraction of their traffic stays in the building, between different colo customers.

Thanks!

Bill
Bill Stewart, bill.stewart@pobox.com

PGP Fingerprint D454 E202 CBC8 40BF 3C85 B884 0ABE 4639

Date: Mon, 20 Nov 2000 00:16:48 -0800
To: cypherpunks@cyberpass.net
From: petro <petro@bounty.org>
Subject: Re: Carnivore All-Consuming

>A totally different bandwidth segment is inside the big hosting centers -
>Exodus, Globalcenter, etc. Most of that's Gigabit Ether,

We've got Fiber running to our cage, but you're right about the Gigabit part.

A quote from Petro's Archives:

**********************************************

"Despite almost every experience I've ever had with federal authority, I keep imagining its competence." John Perry Barlow