OSS Discovery Tips and Tricks

This section is intented to provide informal information on how to best use OSS Discovery. If you've used OSS Discovery in an unusual way or on an unverified platform, or if you've found a particularly groovy workaround for an outstanding issue, let us know and we'll post the information here.

The tips and trips currently documented below are:



Pure Ruby vs. JRuby/Java

We've seen some comments on various blogs related to the size of the download and install. Most of these were from people who downloaded the Full package.

OSS Discovery is very small if you run the pure Ruby version of it...obviously you need Ruby installed on your machine, but after that, it's quite small. The JRuby/Java version on the other hand is large. Some are alarmed with how large it is, but the reason we provide it is so that it can be self-contained and no other packages or prerequisites need to be downloaded.

There's a middle option which is just the JRuby version - this packages OSS Discovery and JRuby and assumes you have a JRE already installed.

So, choose your package - small, pure Ruby, or large-Java-JRuby - depends upon your environment and what you have installed already.



Proxy Support in OSS Discovery (last updated: 5/1/08)

OSS Discovery supports proxies, both authenticated and non-authenticated. We anticipate making it easier to submit scans through a proxy than it is today, but proxies are supported with a little configuration work.

There are a couple of ways to deal with proxies:

  1. set the proxy configuration in the OSS Discovery config.yml file and resubmit scan results
  2. copy and paste the scanresults-census.txt file into the scan post URL

Either approach should work fine. The wiki entry outlines how to configure proxy support in OSS Discovery

How to set the proxy configuration in the config.yml file and resubmit scan results

You can set the proxy configuration and credentials in a configuration file found in:

{OSSDISCOVERY_HOME}/lib/conf/config.yml

(where OSSDISCOVERY_HOME is the directory where OSS Discovery was installed....if you used the windows installer package, it installed in the program files directory on your c drive.

To configure the proxy settings for OSS Discovery, edit the OSSDISCOVERY_HOME/lib/conf/config.yml file. About line 176 you should see a proxy_host attribute:

176 proxy_host: nil # "192.168.10.211"
177 proxy_port: nil # 4545 # 3128
178 proxy_user: nil #
179 proxy_password: nil #

Change the nil values to the hostname, port, and username/password if you have one on your proxy.

At that point you can either rescan or run this command line from the installation directory:

discovery.bat --deliver-results scanresults-census.txt

if you're running windows, or

./discovery --deliver-results scanresults-census.txt

If you're using a *nix variant scan machine.

The command line above will submit the results without needing to rescan the drive if you have already scanned your system but hit a proxy issue in delivering the census scan results.

If You Do Not Know Your Proxy Configuration

If you do not know your proxy configuration, it's still possible to submit scan results. You can open this URL (https://www.osscensus.org/service/upload) in your browser and attach (upload) the scanresults-census.txt file that was created in your OSSDISCOVERY_HOME.

This 2nd approach will use your browser's proxy configuration to post the results and therefore you will not need to know your proxy configuration if your browser is already reaching the internet fine.

We have seen some issues when posting census data when behind an Microsoft NTLM ISA server which we are in the process of tracking down now. However, the 2nd approach would alleviate that issue so it is a good workaround.

[Updated 5/1/08 - In version 2.0.1 of OSS Discovery and later, under JRuby/Java, the proxy priority is set to prefer BASIC, DIGEST, then NTLM in that order. Before, if it encountered an NTML proxy and NTLM authentication failed, it bailed without attempting basic or digest.]

Please feel free to ask questions or report issues if these two alternatives don't work for you.

Pure Ruby and HTTPS through an Authenticated Proxy

We're using the Apache foundation ?HttpClient for our services when running the product under Java/JRuby. We use standard Http net classes in Ruby when running under pure Ruby.

Pure Ruby Http Net classes do not support HTTPS through an authenticated proxy yet (if you know it does, please share, but we have tested and have had no success with proxying https through an authenticated proxy with standard Ruby net classes.)

If you prefer to run the pure Ruby version of OSS Discovery with an authenticated proxy, you can post an HTTP census report but you will need to change the config.yml file attribute destination_url to use http instead of https and also adjust the override_https: false to override_https: true

Otherwise, if you have an authenticated proxy and prefer to use HTTPS for security reasons, then you should use the JRuby version of OSS Discovery.

Microsoft ISA Server Proxy

We have seen some issues when posting census data when behind an Microsoft NTLM ISA server which we are in the process of tracking down now. If you find yourself in behind an ISA server, give the proxy configuration instructions a try, but if you still have problems, use the copy-paste-post method outlined above in the section "If You Do Not Know Your Proxy Configuration."

We'll update this wiki page on proxy support when the issue is understood and/or resolved.



Throttle the Scan Rate of OSS Discovery

By default the scan rate of OSS Discovery is "all out" – as fast as possible. This can affect the CPU utilization for the duration of the scan and can possibly affect other operations on the machine.

You can throttle the scan rate by telling Discovery to take a breather after every X number of files scanned - the effect is after X number of files, it will sleep for a time (default is 2.5 seconds every 300 scanned files.)

You can use throttling defaults or change them in the config.yml file. Edit the config.yml file and look for the attributes:

# Every x number of files, pause for y number of seconds. 
throttle_number_of_files: 300
throttle_seconds_to_pause: 2.5

Then when you scan your box, scan it with the:

--throttle

command line parameter present. Then it won't eat up so many CPU cycles during scanning but will take longer to complete.



Running OSS Discovery with pure Ruby under Mac OS X

The Ruby that ships with Leopard or Tiger will work for many tasks, but because OSS Discovery makes heavy use of OpenSSL, specifically the SHA256 methods, the Ruby that ships with OS X it will not work out of the box for OSS Discovery.

OSS Discovery needs the SHA256 methods introduced in 0.9.8 of OpenSSL. Leopard ships with a 0.9.7 which doesn't contain the SHA256 methods.

Fortunately, most Mac users know about MacPorts. If you are running Mac and don't use MacPorts, it's a great package management tool. Using MacPorts, you can install its Ruby and be able to run OSS Discovery with the SHA256 openssl it needs.

sudo port install ruby

should be all you need to get a native Ruby that will run OSS Discovery on a Mac.



Suppressing Various Types of Output on Windows

To suppress any console output on Windows run the discovery client with:

--human-results NUL:

or alternatively you can redirect your results to the NUL device

> NUL:

To suppress the creation of the machine readable results file use:

--machine-results NUL:


Suppressing Various Types of Output on *nix Machines

To suppress any console output on Windows run the discovery client with:

--human-results /dev/null

or alternatively you can redirect your results to the NUL device

> /dev/null

To suppress the creation of the machine readable results file use:

--machine-results /dev/null


Suppressing Log File Creation on Windows

To suppress the creation of the log file on windows machines, edit the lib/conf/config.yml file and modify the log_device: attribute to say:

log_device: NUL


Suppressing Log File Creation on *nix Machines

To suppress the creation of the log file *nix machines, edit the lib/conf/config.yml file and modify the log_device: attribute to say:

log_device: /dev/null
Get OSS Discovery at SourceForge.net. Fast, secure and Free Open Source software downloads