RS Accuracy Click here to return to the Data & Code Page
Version: 0.96
© Copyright Joseph F. Knight 2001, All Rights Reserved
========================================================
Thank you for using RS Accuracy (Remote Sensing Accuracy.) This file contains the latest information and instructions for RS Accuracy. Click here to download the latest version of the software.
Table of Contents
1. Introduction
2. Installation instructions
3. Usage
4. Input file formats
5. Known Issues
6. Changelog
7. Disclaimer
RS Accuracy computes and analyses error matrices to estimate the accuracy of thematic classifications. The software takes input from text files, containing either map and reference data for reference sites or pre-made matrices, and then produces numerous accuracy estimators for the classification, such as percent overall accuracy, kappa, errors of ommission and comission, user's and producer's accuracies, and more. Please see the Usage section of this document for specific instructions.
The results of the accuracy estimation calculations have been verified, but this version should be considered a beta, or test, release with respect to overall stability. I am committed to improving this software, so please direct questions or bug reports to Joe Knight at Knight.Joe@epa.gov. Note: this software should in no way be construed as an EPA product or as representing EPA. I am solely responsible for this software.
Installation instructions
-------------------------
To install RS Accuracy, you should follow the following simple steps:
1. Copy the all of the distribution files to a location on
your hard drive (e.g. C:\temp).
2. Run the installer by double-clicking on the setup.exe icon.
3. Follow the instructions of the installer software to install the software.
4. Done! RS Accuracy should be installed on your system.
The user interface for RS Accuracy was designed to be simple and functional. A description of the input fields follows:
- Standard Analysis means non-weighted accuracy estimators (as in weighted kappa).
- Weighted Analysis requires an input file that contains weights for each cells of the error matrix. It uses the weighted kappa technique when computing estimators.
- Input file: The file that contains the input data. The data can be in either matrix or raw format (explained below.)
- Output file: This file will be populated with the error matrix and accuracy estimators. You may choose to either overwrite or append to the output file.
- Number of matrices (only visible when Standard Analysis is selected): The number of matrices present in the input file (max. 100).
- Weight file (only visible when Weighted Analysis is selected): The file containing the weights for use in weighted kappa analysis for each cell of the matrix
Input File Formats
-----------------
Example data files are provided in the installation directory of this software. It is essential that the formats be followed, or the software will not process the files. The files must be comma separated with no spaces or blank lines. The data can be in either raw or matrix mode. In raw mode the input file will contain the map and reference classes for each reference point (max 50,000). In matrix mode the input file will contain the cell values of previously computed error matrices (max 100). Multiple matrices can be included in input files by simply appending each matrix (without blank lines or space) after the previous one. The file structures for both types of files consist of a header line and then data lines, as follows:
A. RAW MODE - header line and then each reference point on a separate line in the "map,reference" order. Classes must be represented by numbers.
MatrixName,# reference sites, # classes
map,reference
map,reference
map,reference
For example, for a three class thematic map with 10 reference sites, the input file could look like this (no spaces or blank lines):
MyMatrix,10,3
1,1
1,2
2,2
2,1
2,3
3,1
3,3
2,2
3,3
1,1
B. MATRIX MODE - header line and then cell values of an error matrix - cells are numbered across the rows, as in this example showing a 3x3 error matrix:
1 2 3
4 5 6
7 8 9
MatrixName,# reference sites, # classes
Cell 1
Cell 2
Cell 3
Cell 4
Cell 5
Cell 6
Cell 7
Cell 8
Cell 9
Additional matrices can be appended to either type of data file by simply starting the next header line immediately after the end of the previous one.
If using weighted kappa analysis, the weights for each cell must be provided in a text file that is similar in format to the matrix mode input file, except with no header file. So, for a 3x3 matrix, the weights file would have nine values on separate lines corresponding to the cells in the error matrix.
- The weighted kappa module currently can process only one matrix in each input file.
- The program is limited to 50,000 reference sites per matrix and 100 matrices per input file.
Changelog
---------
v0.96
Fixed potential overflow in confidence interval calculation
v0.95
Added column and row totals to output error matrix
Added confidence intervals to percent accuracy estimates
v0.94
Added multiple matrix stats summary at end of output file
Changed some variable types to prevent overflows
v0.93
Fixed conditional kappa for good
Raised reference points limit to 50,000
Made program window minimizable
Added labels to error matrix axes
v0.92
Fixed potential divide by zero bug in conditional kappa
v0.91b
Fixed integer overflow bug in conditional kappa calculation
v0.9b
First public release
THERE IS NO WARRANTY FOR THIS SOFTWARE, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDER PROVIDES THE SOFTWARE "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE SOFTWARE IS WITH YOU. SHOULD THE SOFTWARE PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING WILL THE COPYRIGHT HOLDER BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE SOFTWARE (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A FAILURE OF THE SOFTWARE TO OPERATE WITH ANY OTHER SOFTWARE), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.