The Darknet refers to networks that are not indexed by search engines such as Google, Yahoo or Bing. These are networks that are only available to a select group of people and not to the general internet public, and only accessible via authorization, specific software and configurations. This includes harmless places such as academic databases and corporate sites, as well as those with shadier subjects such as black markets, fetish communities, and hacking and piracy.
The Darknet is an overlay network to the internet that can only be accessed by specialized software, configurations and special authorizations, and often makes use of non-standard communication protocols in order for it to be deliberately inaccessible by the internet.
The term was originally coined in the 1970s to refer to computer networks that were isolated from the ARPANET for obvious security reasons. These darknets were able to receive communication from the ARPANET but were inaccessible and invisible in network lists and would disregard pings and other regular inquiries.
The term gained popular acceptance after the publication of the paper “The Darknet and the Future of Content Distribution” in 2002. In this paper, four Microsoft employees (Biddle, England, Peinado and Willman) argued that the darknet’s presence is the primary hindrance to the development of workable DRM technologies because of the prospect of inevitable copyright infringement.
The idea of the darknet is based upon three assumptions:
- Any widely distributed object will be available to a fraction of users in a form that permits copying.
- Users will copy objects if it is possible and interesting to do so.
- Users are connected by high-bandwidth channels.
The darknet is the distribution network that emerges from the injection of objects according to assumption 1 and the distribution of those objects according to assumptions 2 and 3.
One implication of the first assumption is that any content protection system will leak popular or interesting content into the darknet, because some fraction of users–possibly experts–will overcome any copy prevention mechanism or because the object will enter the darknet before copy protection occurs.
The term “widely distributed” is intended to capture the notion of mass market distribution of objects to thousands or millions of practically anonymous users. This is in contrast to the protection of military, industrial, or personal secrets, which are typically not widely distributed and are not the focus of this paper.
Like other networks, the darknet can be modeled as a directed graph with labeled edges. The graph has one vertex for each user/host. For any pair of vertices (u,v), there is a directed edge from u to v if objects can be copied from u to v. The edge labels can be used to model relevant information about the physical network and may include information such as bandwidth, delay, availability, etc. The vertices are characterized by their object library, object requests made to other vertices, and object requests satisfied.
To operate effectively, the darknet has a small number of technological and infrastructure requirements, which are similar to those of legal content distribution networks. These infrastructure requirements are:
- facilities for injecting new objects into the darknet (input)
- a distribution network that carries copies of objects to users (transmission)
- ubiquitous rendering devices, which allow users to consume objects (output)
- a search mechanism to enable users to find objects (database)
- storage that allows the darknet to retain objects for extended periods of time. Functionally, this is mostly a caching mechanism that reduces the load and exposure of nodes that inject objects.
The dramatic rise in the efficiency of the darknet can be traced back to the general technological improvements in these infrastructure areas. At the same time, most attempts to fight the darknet can be viewed as efforts to deprive it of one or more of the infrastructure items. Legal action has traditionally targeted search engines and, to a lesser extent, the distribution network. As we will describe later in the paper, this has been partially successful. The drive for legislation on mandatory watermarking aims to deprive the darknet of rendering devices. We will argue that watermarking approaches are technically flawed and unlikely to have any material impact on the darknet. Finally, most content protection systems are meant to prevent or delay the injection of new objects into the darknet. Based on our first assumption, no such system constitutes an impenetrable barrier, and we will discuss the merits of some popular systems.
We see no technical impediments to the darknet becoming increasingly efficient (measured by aggregate library size and available bandwidth). However, the darknet, in all its transport-layer embodiments, is under legal attack. In this paper, we speculate on the technical and legal future of the darknet, concentrating particularly, but not exclusively, on peer-to-peer networks.