Drop te casestudy-argentina article, it is outdated (taks about migrathing
to FreeBSD 4), and it is also clear that they still use FreeBSD any more.
This commit is contained in:
Gavin Atkinson
parent
198eda3b11
commit
cc73bd486c
Notes:
svn2git
2020-12-08 03:00:23 +00:00
svn path=/head/; revision=45689
4 changed files with 0 additions and 341 deletions
en_US.ISO8859-1
|
|
@ -3,7 +3,6 @@
|
||||||
SUBDIR =
|
SUBDIR =
|
||||||
SUBDIR+= bsdl-gpl
|
SUBDIR+= bsdl-gpl
|
||||||
SUBDIR+= building-products
|
SUBDIR+= building-products
|
||||||
SUBDIR+= casestudy-argentina.com
|
|
||||||
SUBDIR+= committers-guide
|
SUBDIR+= committers-guide
|
||||||
SUBDIR+= compiz-fusion
|
SUBDIR+= compiz-fusion
|
||||||
SUBDIR+= console-server
|
SUBDIR+= console-server
|
||||||
|
|
|
||||||
|
|
@ -1,19 +0,0 @@
|
||||||
#
|
|
||||||
# $FreeBSD$
|
|
||||||
#
|
|
||||||
# Article: Casestudy from Argentina.com
|
|
||||||
|
|
||||||
DOC?= article
|
|
||||||
|
|
||||||
FORMATS?= html
|
|
||||||
WITH_ARTICLE_TOC?= YES
|
|
||||||
|
|
||||||
INSTALL_COMPRESSED?= gz
|
|
||||||
INSTALL_ONLY_COMPRESSED?=
|
|
||||||
|
|
||||||
SRCS= article.xml
|
|
||||||
|
|
||||||
URL_RELPREFIX?= ../../../..
|
|
||||||
DOC_PREFIX?= ${.CURDIR}/../../..
|
|
||||||
|
|
||||||
.include "${DOC_PREFIX}/share/mk/doc.project.mk"
|
|
||||||
|
|
@ -1,316 +0,0 @@
|
||||||
<?xml version="1.0" encoding="iso-8859-1"?>
|
|
||||||
<!DOCTYPE article PUBLIC "-//FreeBSD//DTD DocBook XML V5.0-Based Extension//EN"
|
|
||||||
"http://www.FreeBSD.org/XML/share/xml/freebsd50.dtd">
|
|
||||||
<article xmlns="http://docbook.org/ns/docbook" xmlns:xlink="http://www.w3.org/1999/xlink" version="5.0" xml:lang="en">
|
|
||||||
|
|
||||||
<info><title>Argentina.com : A Case Study</title>
|
|
||||||
|
|
||||||
<authorgroup>
|
|
||||||
<author><personname><firstname>Carlos</firstname><surname>Horowicz</surname></personname><affiliation>
|
|
||||||
<address><email>ch@argentina.com</email>
|
|
||||||
</address>
|
|
||||||
</affiliation></author>
|
|
||||||
</authorgroup>
|
|
||||||
|
|
||||||
<legalnotice xml:id="trademarks" role="trademarks">
|
|
||||||
&tm-attrib.freebsd;
|
|
||||||
&tm-attrib.intel;
|
|
||||||
&tm-attrib.redhat;
|
|
||||||
&tm-attrib.general;
|
|
||||||
</legalnotice>
|
|
||||||
|
|
||||||
<pubdate>$FreeBSD$</pubdate>
|
|
||||||
|
|
||||||
<releaseinfo>$FreeBSD$</releaseinfo>
|
|
||||||
</info>
|
|
||||||
|
|
||||||
<sect1 xml:id="overview">
|
|
||||||
<title>Overview</title>
|
|
||||||
|
|
||||||
<para>Argentina.Com is an Argentine ISP with a small infrastructure
|
|
||||||
of fewer than 15 employees and whose primary source of income
|
|
||||||
originates in the free dialup business. It began operation in the
|
|
||||||
year 2000 with barely one server for mail and chat.</para>
|
|
||||||
|
|
||||||
<para>It has since grown to a market presence in the Argentine free
|
|
||||||
dialup market of 4.5 billion minutes annually. Its most popular
|
|
||||||
product provides nearly half a million users with free e-mail with
|
|
||||||
webmail, POP3 and SMTP access, and 300M disk space. Towards the
|
|
||||||
end of 2002 there were around 50,000 mail users. After two and a
|
|
||||||
half years of re-engineering and consistent technical improvements
|
|
||||||
this ISP has grown by a factor of 3 in terms of billing, and by a
|
|
||||||
factor of 10 with regard to the mail user base.</para>
|
|
||||||
|
|
||||||
<para>Our competitors in the Argentine market of free dialup include
|
|
||||||
Fullzero which is owned by the Clarin Media Group, Alternativa
|
|
||||||
Gratis, and Tutopia which is funded by IFX and promoted by
|
|
||||||
Hotmail. Some of these large corporate competitors started their
|
|
||||||
free dialup business with multi-million dollar investments and
|
|
||||||
aggressive television and Internet ad campaigns. Argentina.Com
|
|
||||||
does not rely on advertising like these other larger corporations.
|
|
||||||
It has climbed to the fourth position and to an 8% market share
|
|
||||||
during the last two years thanks to superior quality of service.</para>
|
|
||||||
|
|
||||||
<para>In Argentina and Latin America in general people who do not
|
|
||||||
have computers at home go to so called <quote>Locutorios</quote>
|
|
||||||
(Internet Centers), where for a few pesos they can use a computer
|
|
||||||
connected to the Internet and usually read and write emails
|
|
||||||
through popular webmails like Hotmail, Yahoo or
|
|
||||||
Argentina.Com.</para>
|
|
||||||
|
|
||||||
<para>Due to limited financial resources, Argentina.Com made the
|
|
||||||
decision to invest in a new email system instead of publicity in
|
|
||||||
the media. This strategic decision opens the door to a future
|
|
||||||
business in the corporate and paid email arena.</para>
|
|
||||||
</sect1>
|
|
||||||
|
|
||||||
<sect1 xml:id="challenge">
|
|
||||||
<title>The Challenge</title>
|
|
||||||
|
|
||||||
<para>The main challenge for Argentina.Com is to achieve a dialup
|
|
||||||
uptime of at least 99.95%, or less than 5 hours yearly
|
|
||||||
downtime. Due to the high rotation and volatility in this
|
|
||||||
business, things have to work correctly so the user does not switch
|
|
||||||
-voluntarily or not- the dialup provider or the number he calls to
|
|
||||||
connect. The dialup business involves a support structure to deal
|
|
||||||
with the Telcos about telephony problems and quality of service,
|
|
||||||
plus a technical structure where latency and packet-loss should be
|
|
||||||
minimized due to the UDP nature of Radius and DNS, and where
|
|
||||||
recursive DNS should always be available.</para>
|
|
||||||
|
|
||||||
<para>This also implies having a high uptime in the POP3 and SMTP
|
|
||||||
services, and in the webmail. For POP3 and SMTP we estimated the
|
|
||||||
need for an uptime equal to the one for dialup, whereas for the
|
|
||||||
webmail we could live with 99.5% which means around two days of
|
|
||||||
yearly downtime.</para>
|
|
||||||
|
|
||||||
<para>We decided to migrate the email to a proprietary, opensource
|
|
||||||
architecture which should be horizontally scalable, and whose
|
|
||||||
antivirus and antispam infrastructure should support more than
|
|
||||||
just one type of mailstore or back-end.</para>
|
|
||||||
|
|
||||||
<para>The rough competition in the free email market, mostly due to
|
|
||||||
the recent improvements introduced by Hotmail, Yahoo and Gmail,
|
|
||||||
made it necessary to design the new system with at least 300M user
|
|
||||||
disk space, but at a cost lower than 3 US dollars per GB with some
|
|
||||||
degree of redundancy. Bear in mind that rackmountable hardware is
|
|
||||||
hard to find in Argentina, and is between 30 and 40% more
|
|
||||||
expensive than in the US. Our total budget for equipment
|
|
||||||
acquisition in two years was 75,000 USD, which is only a fraction
|
|
||||||
of our direct competitors' investments.</para>
|
|
||||||
|
|
||||||
<para>With regard to the antispam service, it became necessary to
|
|
||||||
develop a product that could compete with the systems offered by
|
|
||||||
the big ones. Given the hostile conditions imposed by the
|
|
||||||
existence of spam (dictionary attacks, spams with high degree of
|
|
||||||
obfuscation and refinement, phishing, trojans, mail-bombs, etc.)
|
|
||||||
it becomes very difficult to achieve an excellent uptime while
|
|
||||||
repelling attacks. One must also be careful that the user does not
|
|
||||||
lose mails because of false positives in the classification
|
|
||||||
strategy, that he does not become flooded with spam or spam
|
|
||||||
notifications, and dangerous mails do not make it through to his
|
|
||||||
mailbox. In addition, the technical infrastructure for spam
|
|
||||||
classification should not introduce noticeable delays in the
|
|
||||||
delivery of mails. Finally, the mail system has to be protected
|
|
||||||
from spammers who might misuse it to send spam.</para>
|
|
||||||
|
|
||||||
<para>The opensource paradigm tends to require hiring large teams of
|
|
||||||
system administrators, operators and programmers who apply
|
|
||||||
patches, correct bugs and integrate platforms. The opposed
|
|
||||||
paradigm is also costly because of expensive software licences,
|
|
||||||
the need for increasingly expensive hardware and a large support
|
|
||||||
staff. So the challenge was to find the right mixture for scarce
|
|
||||||
human and monetary resources, high stability and predictability,
|
|
||||||
and quick and reliable deployment. In Buenos Aires, well-trained
|
|
||||||
Computer Science professionals are hard to find, most of them live
|
|
||||||
and work abroad, while the remaining have stable jobs either at
|
|
||||||
the government or big companies.</para>
|
|
||||||
|
|
||||||
</sect1>
|
|
||||||
|
|
||||||
<sect1 xml:id="freebsd">
|
|
||||||
<title>The FreeBSD solution</title>
|
|
||||||
|
|
||||||
<sect2 xml:id="freebsd-intro">
|
|
||||||
<title>Introduction</title>
|
|
||||||
|
|
||||||
<para>At the beginning of 2003 we had a CriticalPath mail system
|
|
||||||
running on Solaris x86 plus a Red Hat box for SMTP, Radius and
|
|
||||||
DNS. The DNS and Radius services were constantly down and we
|
|
||||||
were struggling with huge mail queues. There was an attempt to
|
|
||||||
install CriticalPath for &linux; into Red Hat on an &intel; box with
|
|
||||||
a Megaraid card, but the disk latency was enormous and the mail
|
|
||||||
application never really worked.</para>
|
|
||||||
|
|
||||||
<para>The first step depicted towards the "FreeBSD solution"
|
|
||||||
consisted in migrating this hardware and commercial software to
|
|
||||||
FreeBSD 4.8 with &linux; emulation.</para>
|
|
||||||
</sect2>
|
|
||||||
|
|
||||||
<sect2 xml:id="freebsd-choice">
|
|
||||||
<title>The choice of FreeBSD</title>
|
|
||||||
|
|
||||||
<para>The FreeBSD operating system is well-known for its great
|
|
||||||
stability, plus its pragmatism and common sense to put
|
|
||||||
applications on-line thanks to its excellent <link xlink:href="http://www.FreeBSD.org/ports">Ports System</link>. We
|
|
||||||
consider its <link xlink:href="http://www.FreeBSD.org/releng">release
|
|
||||||
engineering process</link> to be easily understandable, while
|
|
||||||
the users' community at the official mailing lists keeps a
|
|
||||||
polite and civilized style when it comes to asking for support
|
|
||||||
or reading other people's problems and solutions.</para>
|
|
||||||
|
|
||||||
<para>Another important feature is quick deployment. Fortunately,
|
|
||||||
we could state our OS install policy around FreeBSD's great
|
|
||||||
out-of-the-box capability. In a small company you sometimes need
|
|
||||||
to run to a Datacenter and quickly setup a server for some
|
|
||||||
service. In the last two years, Argentina.Com acquired around
|
|
||||||
forty servers, most of them Pentium IV but also several
|
|
||||||
double-Xeons and a few double-Opterons to be co-located in the
|
|
||||||
Datacenters where we have dialup and hosting operation
|
|
||||||
contracts. All of them run FreeBSD, ranging from 4.8 (there are
|
|
||||||
a couple with two years uptime and zero trouble) til currently
|
|
||||||
6.0-BETA2.</para>
|
|
||||||
|
|
||||||
<para>The general policy for the operating system is to try to
|
|
||||||
bring all servers periodically to the stable code branch by
|
|
||||||
using <literal>RELENG_4</literal>, <literal>RELENG_5</literal>
|
|
||||||
and now <literal>RELENG_6</literal>. This regularity lets us be
|
|
||||||
more prepared regarding possible exploits at the operating
|
|
||||||
system or base software level, especially in web servers.</para>
|
|
||||||
|
|
||||||
</sect2>
|
|
||||||
|
|
||||||
<sect2 xml:id="freebsd-engineer">
|
|
||||||
<title>Basic re-engineering</title>
|
|
||||||
|
|
||||||
<para>The first re-engineering step was to put in place two
|
|
||||||
FreeBSD 4.8 boxes whose unique task was to be authoritative DNS
|
|
||||||
for all our domains. The chosen software was Bind9. Those boxes
|
|
||||||
were co-located in different datacenters, taking care that there
|
|
||||||
was good latency between them to avoid zone transfer problems,
|
|
||||||
and making it possible to deal with TTLs between 60 and 600
|
|
||||||
seconds to have quicker response in case of trouble.</para>
|
|
||||||
|
|
||||||
<para>Second step was to deploy two more boxes of the same class,
|
|
||||||
again in different Datacenters, to only deal with Radius and
|
|
||||||
recursive DNS. The Network Access Servers at the Telcos were
|
|
||||||
configured to send Radius Authorization and Accounting to those
|
|
||||||
servers, and to assign these recursive DNSs to dialup users.</para>
|
|
||||||
|
|
||||||
<para>The third <quote>golden rule</quote> never to put SMTP
|
|
||||||
incoming and outgoing in the same servers. We deployed separate
|
|
||||||
FreeBSD boxes with postfix for incoming and outgoing mail.</para>
|
|
||||||
|
|
||||||
</sect2>
|
|
||||||
|
|
||||||
<sect2 xml:id="freebsd-email">
|
|
||||||
<title>Email migration</title>
|
|
||||||
|
|
||||||
<para>The email migration required careful planning due to the
|
|
||||||
fact that we were going to migrate both mail front and
|
|
||||||
back-ends. We first built a perimetral antispam and antivirus
|
|
||||||
system in FreeBSD 4.x and 5.x based on postfix, amavisd-new,
|
|
||||||
clamav and SpamAssassin. These systems were to deliver mails to
|
|
||||||
both the old and the new system until the new back-end was in
|
|
||||||
place. In the meantime, we added small FreeBSD NFS boxes to
|
|
||||||
increase CriticalPath's mailspool, without any problem.</para>
|
|
||||||
|
|
||||||
<para>At the frontline of incoming mail, we put in place several
|
|
||||||
MXs of the Argentina.com domain to filter dictionary attacks
|
|
||||||
(attempts to forward mail to nonexistent users) as well as a
|
|
||||||
black-list derived from SURBL that resulted in almost no false
|
|
||||||
positives. The mails are then multiplexed to a cluster of
|
|
||||||
double-Xeons and double-Opterons where we run amavisd-new with
|
|
||||||
MySQL based white and black-listing. We discarded the use of
|
|
||||||
Bayes and Autowhitelisting at the global level because of great
|
|
||||||
quantities of false positives and false negatives. We instead
|
|
||||||
defined a few spam levels going from the least to the most
|
|
||||||
tolerant, each one with cutoff or discard levels. Every email
|
|
||||||
with a score below the one associated with the selected spam
|
|
||||||
tolerance goes to the user's Inbox. Emails between this level
|
|
||||||
and the cutoff level go to a user's folder named Spam, and those
|
|
||||||
above the cutoff level get discarded because it is a very obvious
|
|
||||||
spam. For the sake of simplicity, we transparently associated
|
|
||||||
the use of the Address Book with the antispam system, so that
|
|
||||||
every personal contact gets automatically whitelisted.</para>
|
|
||||||
|
|
||||||
<para>With the introduction of Spamassassin 3.x, the DNS traffic
|
|
||||||
to query global blacklists grew considerably, so we signed
|
|
||||||
agreements with SpamCop, Spamhaus and SURBL to install public
|
|
||||||
mirrors of their databases in our FreeBSD equipment. Thanks to
|
|
||||||
these mirrors that cost us between 1 and 2Mbps in traffic, we
|
|
||||||
were able to dramatically cut down Spamassassin latency.</para>
|
|
||||||
|
|
||||||
<para>At the 3rd level there is the delivery to the maildrops. As
|
|
||||||
soon as we started building a new Cyrus-Imap back-end with MySQL
|
|
||||||
authentication, we needed to multiplex incoming mail to users in
|
|
||||||
both old and new maildrop formats. Finally, we managed to
|
|
||||||
migrate hundreds of thousands of mailspools to the new Cyrus
|
|
||||||
architecture using a great tool named imapsync, which is
|
|
||||||
directly installable from ports. We also put perdition, a POP3
|
|
||||||
and IMAP proxy, in the middle to assure a transparent migration
|
|
||||||
and distribution of mailboxes across several servers. Briefly,
|
|
||||||
all information of where a user's maildrop is located resides in
|
|
||||||
MySQL, and is being used by all software pieces in the
|
|
||||||
chain.</para>
|
|
||||||
|
|
||||||
<para>With regard to the hardware for disk space, we currently use
|
|
||||||
seven Cyrus-Imap loaded FreeBSD boxes with diverse hardware. The
|
|
||||||
biggest are Pentium IV with 4G of RAM and 3ware cards in chassis
|
|
||||||
with 12 hotswappable bays, organized in 3 RAID-5 units of 1
|
|
||||||
Terabyte each. The 3ware software sends you en email whenever
|
|
||||||
the RAID is degraded -mostly because of a failing disk- and lets
|
|
||||||
you rebuild the RAID with everything up and running. We use
|
|
||||||
smartmontools in the cases where we have less redundancy, to
|
|
||||||
have immediate alerts of disks with temperature problems or
|
|
||||||
failing selftests.</para>
|
|
||||||
|
|
||||||
<para>As webmail software, we chose a commercial product named
|
|
||||||
Atmail, which is available with perl sources and utilizes
|
|
||||||
mod_perl. Under FreeBSD it is extremely easy to deal with perl
|
|
||||||
modules, you do not even need to use the CPAN shell, you just
|
|
||||||
have to choose the right port and run "make install". After
|
|
||||||
several months of integration work, we integrated the
|
|
||||||
Client-only version of Atmail that talks IMAP with our
|
|
||||||
back-ends. We had to modify some parts of the code to adapt the
|
|
||||||
product to our massive free environment, and to our antispam and
|
|
||||||
antivirus perimeter, in addition to our specific customizations
|
|
||||||
and translations.</para>
|
|
||||||
|
|
||||||
</sect2>
|
|
||||||
|
|
||||||
<sect2 xml:id="freebsd-web">
|
|
||||||
<title>Web migration</title>
|
|
||||||
|
|
||||||
<para>With the adoption of FreeBSD, there was almost no additional
|
|
||||||
effort necessary to setup a working Apache, PHP and MySQL
|
|
||||||
environment in minutes. Even the upgrades from PHP4 to PHP5 were
|
|
||||||
painless. The ports system was again extremely useful in these
|
|
||||||
cases, and permitted us to do things like compress text and html
|
|
||||||
contents in Apache with just a few lines of documentation. In
|
|
||||||
addition, we have experienced excellent performance and
|
|
||||||
rock-solid stability and uptime.</para>
|
|
||||||
</sect2>
|
|
||||||
|
|
||||||
</sect1>
|
|
||||||
|
|
||||||
<sect1 xml:id="results">
|
|
||||||
<title>Results</title>
|
|
||||||
|
|
||||||
<para>We managed to deploy a FreeBSD based email architecture that
|
|
||||||
is horizontally scalable, using 3 Terabyte &intel; based storage
|
|
||||||
servers at a current cost of 3 dollars per Gigabyte with
|
|
||||||
redundancy.</para>
|
|
||||||
|
|
||||||
<para>The great stability achieved enabled Argentina.Com to explore
|
|
||||||
other fields like hosting for resellers and housing with presence
|
|
||||||
in three Argentine Datacenters.</para>
|
|
||||||
|
|
||||||
<para>We offer now also corporate dialup for roaming users in
|
|
||||||
Argentina and Peru thanks to our presence and contracts with most
|
|
||||||
Telcos. Among our indirect customers, there are major American
|
|
||||||
companies like Ford, Exxon and Reuters. We now run the free dialup
|
|
||||||
business in Brazil, Chile, Colombia and Panama as well.</para>
|
|
||||||
</sect1>
|
|
||||||
|
|
||||||
</article>
|
|
||||||
|
|
@ -100,11 +100,6 @@
|
||||||
Products with FreeBSD</a> (building-products)<br/>
|
Products with FreeBSD</a> (building-products)<br/>
|
||||||
How FreeBSD can help you build a better product.</p>
|
How FreeBSD can help you build a better product.</p>
|
||||||
|
|
||||||
<p><a
|
|
||||||
href="&url.articles;/casestudy-argentina.com/index.html">Argentina.com:
|
|
||||||
A Case Study</a> (casestudy-argentina.com)<br/>
|
|
||||||
How FreeBSD helped a large ISP in Latin America.</p>
|
|
||||||
|
|
||||||
<p><a href="&url.articles;/committers-guide/index.html">The
|
<p><a href="&url.articles;/committers-guide/index.html">The
|
||||||
Committer's Guide</a> (committers-guide)<br/>
|
Committer's Guide</a> (committers-guide)<br/>
|
||||||
Introductory information for FreeBSD committers.</p>
|
Introductory information for FreeBSD committers.</p>
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue