diff --git a/en_US.ISO8859-1/articles/Makefile b/en_US.ISO8859-1/articles/Makefile index f26f48dbed..c37b5ef420 100644 --- a/en_US.ISO8859-1/articles/Makefile +++ b/en_US.ISO8859-1/articles/Makefile @@ -38,7 +38,6 @@ SUBDIR+= serial-uart SUBDIR+= solid-state SUBDIR+= vinum SUBDIR+= vm-design -SUBDIR+= wp-toolbox # ROOT_SYMLINKS+= new-users diff --git a/en_US.ISO8859-1/articles/wp-toolbox/Makefile b/en_US.ISO8859-1/articles/wp-toolbox/Makefile deleted file mode 100644 index bf453e2a12..0000000000 --- a/en_US.ISO8859-1/articles/wp-toolbox/Makefile +++ /dev/null @@ -1,19 +0,0 @@ -# -# $FreeBSD$ -# -# Article: Casestudy about FreeBSD as a Software Testing Toolbox - -DOC?= article - -FORMATS?= html -WITH_ARTICLE_TOC?= YES - -INSTALL_COMPRESSED?= gz -INSTALL_ONLY_COMPRESSED?= - -SRCS= article.xml - -URL_RELPREFIX?= ../../../.. -DOC_PREFIX?= ${.CURDIR}/../../.. - -.include "${DOC_PREFIX}/share/mk/doc.project.mk" diff --git a/en_US.ISO8859-1/articles/wp-toolbox/article.xml b/en_US.ISO8859-1/articles/wp-toolbox/article.xml deleted file mode 100644 index e9a6f1a294..0000000000 --- a/en_US.ISO8859-1/articles/wp-toolbox/article.xml +++ /dev/null @@ -1,449 +0,0 @@ -<?xml version="1.0" encoding="iso-8859-1"?> -<!DOCTYPE article PUBLIC "-//FreeBSD//DTD DocBook XML V5.0-Based Extension//EN" - "http://www.FreeBSD.org/XML/share/xml/freebsd50.dtd" [ -<!ENTITY ports "Ports Collection"> -<!ENTITY os.ports "{{{os}}} {{{ports}}}"> -<!ENTITY frisbee "<application xmlns='http://docbook.org/ns/docbook'>Frisbee</application>"> -<!ENTITY ghost "<application xmlns='http://docbook.org/ns/docbook'>Ghost</application>"> -<!ENTITY nessus "<application xmlns='http://docbook.org/ns/docbook'>Nessus</application>"> -]> -<article xmlns="http://docbook.org/ns/docbook" xmlns:xlink="http://www.w3.org/1999/xlink" version="5.0" xml:lang="en"> - - <info><title>Creating a Software Testing Environment Using &os;</title> - - <authorgroup> - <author><personname><firstname>Chris</firstname><surname>McMahon</surname></personname><affiliation> - <address><email>christopher.mcmahon@gmail.com</email></address> - </affiliation></author> - </authorgroup> - - <legalnotice xml:id="trademarks" role="trademarks"> - &tm-attrib.freebsd; - &tm-attrib.intel; - &tm-attrib.microsoft; - &tm-attrib.symantec; - &tm-attrib.general; - </legalnotice> - - <pubdate>$FreeBSD$</pubdate> - - <releaseinfo>$FreeBSD$</releaseinfo> - - <abstract><para>A slightly altered version of this paper was first - published in the Pacific Northwest Software Quality Conference - Proceedings, 2005.</para></abstract> - - </info> - - - <sect1 xml:id="overview"> - <title>Overview</title> - - <para>From late 2003 until early 2005, I was a tester in an - all-&windows; environment. Although unlikely on the face of it, - &os; became a valuable test tool platform in that context. - &os; contains useful and powerful applications for any tester - in any environment.</para> - - <para>Unlike Linux, &os; is a single monolithic project, rather - than a collection of disparate parts assembled into a - distribution. And the most attractive part of &os; for a - software tester is the &os.ports;—a very large, - managed set of software applications with a single, simple, and - uniform installation procedure.</para> - - <para>This paper describes several software test tools from the - &os.ports; that I used to test software and - systems in an all-Windows environment.</para> - - </sect1> - - <sect1 xml:id="challenge"> - <title>The Challenge</title> - - <para>Software testing environments are radically more complex - than software development environments. Interconnected systems - to test, network entities, databases, and filesystems present - challenges to testers that developers can, for the most part, mock - out and essentially ignore. Software testers need more tools, - and more complex tools, than do software developers.</para> - - <para>On the other hand, software development tools are much more - highly evolved than software testing tools. There is no Eclipse, - or IntelliJ, or even Visual Studio aimed at testing. Testers - struggle and scratch to find tools appropriate to their test - environments and appropriate to their Systems Under Test (SUTs).</para> - - <para>Every test environment is different. Finding appropriate - test tools is challenging. And when a tester has identified a - set of useful tools, introducing them to the test environment - and integrating them with the test environment is a challenge.</para> - - </sect1> - - <sect1 xml:id="freebsd-solution"> - <title>The &os; Solution</title> - - <sect2 xml:id="freebsd-intro"> - <title>Introduction</title> - - <para>The set of tools available with the &os; Operating - System is amazing. The &os; <link xlink:href="http://www.freebsd.org/ports">&ports;</link> - contains more than thirteen thousand separate applications, - all of which have a standard installation procedure and - conform to a set of guidelines that make them reliable without - the need to manage dependencies, appropriate versions, and all - of the other problems that affect even the most well-managed - Linux distribution or the various versions of Microsoft - Windows. The monolithic nature of &os; and the &os.ports; - removes much of the trouble of integrating - tools with the test environment, regardless of the OS under - which the SUT runs. &os; is a highly evolved server - environment, and contains so many reliable applications, that - every tester should consider adding a &os; machine (or - several) to their test environment.</para> - - <para>Of course, all of the applications available in the - &os.ports; will not be appropriate for any single - test environment. Some of the obvious choices for software and - systems testing are the six hundred or so system utilities, - the more than one thousand network tools, and the fifty-odd - benchmarking tools. Whether your test environment is Windows, - UNIX, Linux, Mac OS, &os; itself, or some combination of - any of them, &os; and the &os.ports; is a - great place to look first.</para> - - </sect2> - - <sect2 xml:id="freebsd-ports"> - <title>How To Use The Ports System</title> - - <para>Installing an application from the &os.ports; is a simple matter of: </para> - - <screen>&prompt.root; <userinput>cd /usr/ports/foo</userinput> -&prompt.root; <userinput>make install</userinput></screen> - - <para>and the system does the rest. It reports build status and - test status, and installs all the relevant documentation as - well. This aspect of &os; is very attractive to a tester, - who typically is pressed for time!</para> - </sect2> - - <sect2 xml:id="freebsd-testing"> - <title>&os; For Testing</title> - - <para>The test environment should be more stable than the SUT. - Once the tester decides to use the tools available on &os;, - &os;'s long record of reliability makes it an easy choice - for a test tools platform.</para> - - <para>My own introduction to &os; occurred when I was hired - by a major vendor of large-scale network security video - services to be their network-testing person in an all-Windows - environment. My first assignment was to replace the obsolete, - buggy, disk imaging system. I chose to do that with an Open - Source disk imaging system called <link xlink:href="http://www.cs.utah.edu/flux/papers/frisbee-usenix03-base.html">&frisbee;</link> - which was implemented originally on &os;. I built the - system—a feature-for-feature replacement for an expensive - proprietary system—but we never actually used it in our - production system.</para> - - <para>In the meantime, I had discovered the &os.ports; - and started to use some of those tools for testing; - and I had discovered the power of disk imaging with &frisbee;, - especially for smoke testing and installation testing; and - &os; became a permanent part of my test lab. The test lab - I built, and the &os; systems I created still exist, and - still provide value to the testers there.</para> - - </sect2> - - <sect2 xml:id="freebsd-collab"> - <title>&os; For Collaboration: Twiki</title> - - <para>A wiki is a simple set of web pages to allow many users to - share information and collaborate on any sort of documents. - <link xlink:href="http://twiki.org">Twiki</link> is a wiki fine-tuned for - document management. It has built-in version control and - security implemented at the user/password level. I used it - for requirements management and traceability.</para> - - <para>The company I worked for managed their requirements in a - proprietary tool with a truly byzantine hierarchical - structure. Although the tool was fine for generating - requirements, it was horrible for testing. Also - unfortunately, the tool supported very few export mechanisms. - From the proprietary tool, I had to:</para> - - <itemizedlist> - <listitem><simpara>Export to Access</simpara></listitem> - <listitem><simpara>Export from Access to Excel</simpara></listitem> - <listitem><simpara>Export from Excel to a delimited file</simpara></listitem> - <listitem><simpara>Import the delimited file to the wiki by means of a Perl script</simpara></listitem> - </itemizedlist> - - <para>And in Twiki I was able to implement version control, - traceability, and test coverage, all features missing from the - proprietary tool. This gave me a remarkable ability to trace - requirements coverage with my tests. Having imported a - complex hierarchy of sets of requirements, I could use Twiki's - simple markup features to show each requirement as tested, - passed, or failed. I could attach test documentation to wiki - pages, and since I had already scripted the process, it was - easy to update the requirements as they changed. And Twiki - itself handled version control for such updates.</para> - - <para>As with all of the examples in this paper, installing - Twiki on &os; is fairly simple. It takes just a few - minutes on a &os; system. However, if you want to use Twiki - on a Microsoft Windows platform, I strongly suggest you read - the Twiki documentation extremely carefully. I know someone - who installed Twiki on Windows, and it took him several days. - Twiki on Windows requires not only knowledge of Windows, but - also deep knowledge of Cygwin and Perl.</para> - - <para>Furthermore, at one point in the project I had to migrate - my wiki from a machine running &os; 4.8 to one running - &os; 5.3. The migration consisted merely of installing - Twiki on &os; 5.3; using <command>tar</command> on the &os; 4.8 - machine to gather all of the Twiki data files specific to my - testing; FTPing the gathered files to the new &os; 5.3 - machine; and untarring the file. The complete set of Twiki - documents migrated with no issues or problems at all. That is - the power of a unified system like &os;.</para> - </sect2> - - <sect2 xml:id="freebsd-frisbee"> - <title>&os; For Disk Imaging: Frisbee</title> - - <para>A disk imaging system is a mechanism for saving and - restoring all of the data on a physical disk. The most - popular commercial system for doing this is probably the - product &ghost;™ from Symantec.</para> - - <para>The &frisbee; enterprise disk imaging system mentioned above - had a lot of features I never implemented in the test lab. - Using &frisbee; and an Open Source tool called <application>PXELINUX</application>, I was - able to:</para> - - <itemizedlist> - <listitem><simpara>Boot the Windows client machines from the network</simpara></listitem> - <listitem><simpara>Make an image of the client</simpara></listitem> - <listitem><simpara>Update the BIOS on the client</simpara></listitem> - <listitem><simpara>Lay down an existing image on the client machine</simpara></listitem> - <listitem><simpara>Make a set of restore CDs for the client</simpara></listitem> - </itemizedlist> - - <para>In the test lab, I only needed to boot from the &frisbee; CD, - make an image, or lay down an image on the client machine. - Both &frisbee; and proprietary imaging systems allow the user to - image individual drives on the client, but I never had a need - to do this.</para> - - <para>Installation testing was a large part of my duties at the - company where I used &os;. To do this testing, I would - typically use &frisbee; to make an image of a machine containing - only a Windows OS, install the SUT, and run a smoke test. The - smoke test typically left the test machine in a very bad - state. But instead of having to painstakingly clean up the - mess left by the failed installation, I simply re-imaged the - machine in question with the bare OS image and started over. - A typical re-image containing only the Windows OS and a few - test tools took less than three minutes. Using &frisbee;, we - could run smoke tests on about six builds per day; before - &frisbee;, we could run smoke tests on about three builds per - week.</para> - - <para>Of course, &ghost; or other proprietary tools also image - machines quickly under these circumstances: once you buy the - tool, license the software, install it on an appropriate - server, and configure it properly. I prefer &frisbee; to &ghost; - because: &frisbee; is marginally faster; &frisbee; is very easy to - install on &os;; and &frisbee; is very efficient. Adding a - couple of small Perl scripts to the normal &frisbee; - distribution gave me an imaging environment tailored for the - test lab.</para> - - <para>I also used &frisbee; to preserve the state of a machine - after I had uncovered particularly complex defects. That is, - if it took a large effort (many steps and/or a long duration - of time) to demonstrate a defect, I could make an image of the - machine at the point at which the defect was visible, and - restore the image at will to demonstrate the defect to - developers or managers.</para> - </sect2> - - <sect2 xml:id="freebsd-nessus"> - <title>&os; Security Testing: &nessus;</title> - - <para>Whenever you have more than one entity on a network, and - whenever you expose a server to the wider Internet, security - of the machine itself is always a concern. <link xlink:href="http://www.nessus.org">&nessus;</link> is an Open Source - remote vulnerability scanner for security and penetration - testing that consistently is rated among the top products of - its type throughout the security industry.</para> - - <para>&nessus; probes a remote machine over the network for - security vulnerabilities. It does a port scan, finds which - ports are open, and investigates the software that has those - ports open for a huge number of security risks, for all major - OSs. It generates detailed reports in a number of formats - that anyone can understand. The number of security probes - available in the default installation of &nessus; is very large, - but sophisticated security and penetration testers take - advantage of NASL, the &nessus; Attack Scripting Language, to - craft their own attacks using &nessus;' available features.</para> - - <para>Of interest is that, while &nessus; is a free download for - UNIX-like systems (and is available in the &ports; of - &os;), it is available on Windows only as a commercial - product from a company called <emphasis>Tenable</emphasis>. The Tenable product is - <application>NeWT</application>, <quote>Nessus on Windows Technology</quote>.</para> - </sect2> - - <sect2 xml:id="freebsd-network"> - <title>&os; Network Tools</title> - - <para>&os; is most widely used as a robust server platform. - It follows, then, that tools related to network analysis and - performance will be highly evolved on &os;. Here is a - brief description of network diagnostic tools that I found - invaluable in testing in a networked environment.</para> - - <para>From the name, one would assume that <link xlink:href="http://www.ntop.org">ntop</link> emulates the functions of - the UNIX &man.top.1; command, but for the network - rather than for the local machine. Perhaps the first version - did; currently, <application>ntop</application> is capable of providing detailed - information about a huge number of hosts and their status and - activities on the network.</para> - - <para>For testing, two features I found very powerful: at a high - level, <application>ntop</application> shows the amount of network - traffic on the entire network segment minute-by-minute, - hour-by-hour, and day-by-day in a graphical format. Also, - <application>ntop</application> shows information about recent connections between - individual hosts on the network.</para> - - <para>It is easy to see traffic trends on the network as they - are occurring; also, if something anomalous appears, <application>ntop</application> - records detailed information about network connections between - hosts, including the ports over which the connection happened. - This was critically important when analyzing software - issues. If <application>ntop</application> showed a period of time for which traffic was - particularly high, I would find out which host was generating - the traffic. I would examine the software running on that - host, over that port. Often it was a new build with a - bug.</para> - - <para><link xlink:href="http://ettercap.sourceforge.net">Ettercap</link> is - a tool for ARP poisoning which can also decipher passwords on - the fly and corrupt IP traffic by means of a Man In The Middle - (MITM) attack. However, I used <application>Ettercap</application> as a performance tool. - In my test labs, all of my &os; machines ran on discarded - hardware, Pentium II processors. I found that when I used - <application>Ettercap</application> to sniff traffic between two hosts, the lack of - processing power caused <application>Ettercap</application> on the slow MITM machine to - start dropping packets, making it look to the client machine - in the SUT as if there was interference or other trouble on - the network. And by varying the load on the &os; machine, - I could in fact control the number of packets being dropped: - running <application>Ettercap</application> and the UNIX <command>yes</command> utility - caused 100% packet loss.</para> - - <para>This was my most creative use of a &os; tool for - testing. In a more straightforward application, any time a - tester needs to eavesdrop on traffic between two hosts on a - network, <application>Ettercap</application> is an excellent choice because of its power - and ease of use.</para> - - <para>Perl gets a special mention here because Perl's network - utilities are outstandingly good compared to other - languages. Perl <literal>Net::*</literal> modules and - <literal>IO::Socket::*</literal> modules are robust and - powerful—but they often fail to compile on Windows. It is - the ease of use of Perl's network utilities on &os; that - gets Perl the mention in this section.</para> - - <para>I use Perl's network utilities to impersonate network - clients and servers for test purposes. On one occasion, I was - required to test software that was a client to an interface on - the <emphasis>New York Stock Exchange</emphasis>. Unfortunately, the NYSE test - server was down about nine days out of ten. I wrote a little - network server in Perl to emulate simple functions of the NYSE - server in order to test the client software.</para> - - <para>On another occasion, I had to test some functions on a web - server. It was not difficult to write a simple Perl HTTP - client to validate that the server was functioning properly.</para> - - <para>I have also used Perl to validate the output from a server - sending to a multicast address. I wrote a simple Perl - multicast client on &os; to monitor the traffic on a - multicast address. Lincoln Stein's excellent - <literal>IO::Socket::Multicast</literal> module made it easy. - (Note: I never got <literal>IO::Socket::Multicast</literal> to - compile on Windows. I tried it on Windows 2000 and on Windows - XP.)</para> - </sect2> - </sect1> - - <sect1 xml:id="conclusion"> - <title>Conclusion</title> - - <para>I used tools from the &os.ports; in four - areas: in the network, where the operating system has very - little impact on how software behaves; for remote security - testing and performance testing in order to manipulate remote - machines over the network, regardless of the operating system; - for disk imaging of Windows, Linux, and &os; machines; and on - the webserver, where Twiki was my collaboration tool of choice.</para> - - <para>Because the installation procedure for all of these tools is - standard, I spent relatively little time installing and - configuring my tools. Because all the tools were hosted on a - single platform, I had all of my configuration and diagnostic - information located in just a few places. I kept all of my - potentially dangerous security tools on a single machine, which - made my presence on the network tolerable to the company's - network management staff. And the compatibility between &os; - versions made it fairly simple to upgrade and to manage multiple - &os; machines. And of course, I could rely on the - correctness of my test results, because the system itself is so - reliable.</para> - - <para>I have tried using Linux in a similar way, but my experience - is that package management quickly becomes tedious if not - overwhelming. The &os.ports; handled that for me. - And many of these tools are simply not available on Microsoft - Windows. And when they (or their equivalents) are available, - their cost, both financial and in terms of overhead, was simply - too high.</para> - - <para>&os;'s simple installation procedures and robust &ports; - makes it easy to experiment with the huge number of - tools available. I often find myself browsing the &ports; - looking for interesting applications to install, just - to see how they work. (I found <application>Ettercap</application> by browsing the &ports;, - a tool that became very useful very quickly.) It - became clear that the more tools I used on &os;, the more - economical became the management of those tools.</para> - - <para>The next time you need to reach into your toolbox for some - sophisticated, reliable, and powerful testing tools, I hope you - find them in &os;.</para> - </sect1> - -<!-- -<sect1> - <title>Biography</title> - - <para>Chris McMahon has been for nearly a decade a tester of library - software, telecom, stock trading, and networked video systems. He - has published a number of articles aimed at technical testers, but - his true career is at being an enthusiastic beginner. Chris works - for ThoughtWorks, Inc. as a tester, and can be reached at - christopher.mcmahon@gmail.com.</para> -</sect1> ---> - -</article> diff --git a/en_US.ISO8859-1/htdocs/marketing/index.xml b/en_US.ISO8859-1/htdocs/marketing/index.xml index fe33145add..390593d099 100644 --- a/en_US.ISO8859-1/htdocs/marketing/index.xml +++ b/en_US.ISO8859-1/htdocs/marketing/index.xml @@ -31,8 +31,6 @@ <li><a href="&url.articles;/linux-comparison/article.html">FreeBSD: An Open Source Alternative to Linux</a></li> <li><a href="&url.articles;/explaining-bsd/article.html">Explaining BSD</a></li> - - <li><a href="&url.articles;/wp-toolbox/article.html">Creating a Software Testing Environment Using FreeBSD</a></li> </ul> <a name="presentations"></a> diff --git a/share/xml/urls.ent b/share/xml/urls.ent index d9d76dbf81..8958d13eab 100644 --- a/share/xml/urls.ent +++ b/share/xml/urls.ent @@ -120,5 +120,3 @@ <!ENTITY url.articles.solid-state.en "&url.doc.langbase.en;/articles/solid-state"> <!ENTITY url.articles.vm-design "&url.doc.langbase;/articles/vm-design"> <!ENTITY url.articles.vm-design.en "&url.doc.langbase.en;/articles/vm-design"> -<!ENTITY url.articles.wp-toolbox "&url.doc.langbase;/articles/wp-toolbox"> -<!ENTITY url.articles.wp-toolbox.en "&url.doc.langbase.en;/articles/wp-toolbox">