Evaluator
=========

This tool reads classification results and creates 2D plots (ROC, DET, Precision/Recall, ...).

This tool uses GNUPLOT for creating the plots. 
Go to http://www.gnuplot.info/download.html for more information on GNUPLOT.


Use
===

evaluator -i configFileName -l logFileName 


-i configFileName 	the xml configuration file - the format is described further
-l logFileName 		the name of a html log file with the liks to the plots
			The plots are stored in a subdirectory of the directory where the log is stored.
			The name of the subdirectory is the same as the name of the log file.



Format of the configuration file:
=================================

<EvaluatorConfig>
	<TestName name="test dataset"/>
	<TestName name="training dataset"/>	
	<TestName name="XXX dataset"/>

	<Classifier name="ClassifierXXX">
		<Test fileName="independentTest"/>
		<Test fileName="testOnTrainigData"/>
		<Test fileName="testx"/>
	</Classifier>
</EvaluatorConfig>			


- there can be multiple classifiers and multiple test for single classifier (usually on the same data)
- the number of test must be the same for all classifiers
- the names of the test must be specified in the <TestName> nodes
- keep the order of the test the same as the order of <TestName> nodes



Format of the file with classification results
==============================================

- the format is purely text
- each row represents a sample
- first two columns are the ground truth - the real probability that the sample belongs to the class 1 and 2
- second two columns are the classification results - again some measure of how probable the sample belongs to class 1 and 2


example:

1	0	0.629141	0.370859
1	0	0.677355	0.322645
0	1	0.307085	0.692915
0	1	0.461971	0.538029




-- There is an example in the ./bin/ directory
There is a configuration file, batch file running the evaluation and some files with classification results/
Just compile the evaluator and run the batch file.
