Usenet in Depth: Retention

Retention is one of the most important features for a usenet provider. Simply put, retention is the number of usenet articles that a server stores and makes available to users. Each usenet server has a set amount of storage, much like a home computer. The amount of usenet article data that is able to be stored is the retention, which is usually expressed as a number of days. 650 days of binary retention means that the hardware is capable of storing 650 days worth of binary articles.

Usenet RetentionAll of the usenet posts made each day are collectively known as  the “usenet feed”. The usenet feed is constantly growing as more and more articles are posted daily. Depending on whom you ask, the usenet feed is currently around 4-6 TB per day, which is a staggering amount of data. The hardware costs for a redundant, secured system to handle that kind of storage is a massive capital expense and the primary reason that usenet providers simply can’t have unlimited retention, or keep all articles forever.

There are two primary types of retention that customers should be concerned with, text group retention and binary group retention. Different types of groups offer different amounts of retention because the storage needs are significantly different. The average message size in a binary group is much larger than the average message size in a text group, not to mention the larger number of articles posted to a binary group. As of this writing NewsgroupDirect offers around 4.5 years of text retention and nearly 2 years of binary retention.

One downside to the increasing retention in the usenet world is in how articles are accessed. When retention was much lower it was possible to simply download all of the headers for a given group and then decide what messages you wanted to download or open. Now, just retrieving the headers for some groups can take several hours. As retention grows it becomes increasingly difficult to manage within a newsreader. Some modern usenet additions, such as message-id searching, have allowed newsreaders to circumvent the large amount of data now present.

Comments are closed.