TedEval: An Architecture for Cross-Experiment Parse Evaluation

Installing TedEval

The latest version of TedEval is available from the TedEval download page.

Running TedEval requires a Java Runtime Environment (JRE). You need version JRE 1.6 or later version. If you don't have JRE installed on your system, download and install the latest version of Java Runtime Environment.

To install TedEval, unpack the TedEval distribution TedEval-1.0.zip or TedEval-1.0.tar.gz by running one of the following commands:

Alternative 1

prompt> tar -zxvf TedEval-1.0.tar.gz

Alternative 2

prompt> gunzip TedEval-1.0.tar.gz
prompt> tar -xvf TedEval-1.0.tar

Alternative 3

prompt> unzip TedEval-1.0.zip

The TedEval Distribution

The distribution of TedEval contains the following files and directories:

LICENSELicense information
READMEReadme file
tedeval.jarExecutable jar file
build.xmlAnt build script
bin/Binaries of TedEval
src/Java source of TedEval
docs/User guide and TedEval documentation

Building TedEval

TedEval is distributed both in binary form and with all source code and documentation. If you want to use TedEval you only need to install the distribution (follow the installation guidelines above), but if you need to build the system you should follow these guidelines. To be able to build TedEval you need Java SE Development Kit (JDK) 6 and Apache Ant 1.7.0. If you don't have these tools please download and install them by following the instructions on their web sites.

To build the TedEval distribution:

prompt> cd <path to TedEval-1.0 directory>
prompt> ant all
<path to TedEval-1.0 directory> should be replaced by the actual path to the directory where TedEval-1.0 is installed.

Testing the Installation

After installing or building TedEval, you can run it to verify that it is working properly on your system.

To run TedEval type the following at the command line prompt (it is important that you are in the TedEval-1.0 installation directory):

prompt> java -jar tedeval.jar 
If TedEval displays something like the message below, the distribution has been installed successfully.
                               TedEval 1.0
                          UniPar Development Team
                            Uppsala University

  java -jar tedeval.jar -h for more help and options

help                      ( -h) : Show options
version                   ( -v) : Show version
Single experiment evaluation:
gold_file                  (-g) : path to gold-parses file (single experiment)
parsed_file                (-p) : path to parse-hypotheses (single experiment)
output_file                (-o) : path to result file (single experiment), default file name is eval

Pairwise Experiment evaluation:
first_gold_file           (-g1) : path to gold-parses file (the first of two experiments)
first_parsed_file         (-p1) : path to parse-hypotheses (the first of two experiments)
second_gold_file          (-g2) : path to gold-parses file (the second of two experiments)
second_parsed_file        (-p2) : path to parse-hypotheses (the second of two experiments)
first_result_file         (-o1) : path to result file (the first of two experiments), default file name is eval1
second_result_file        (-o2) : path to result file (the second of two experiments), default file name is eval2
file_format           (-format) : the default is labeled ptb-like bracketed format (ignoring anything after the dash)
conll                             the conll-x format
bracketed                         labeled ptb-like bracketed format (empty elements not allowed, ignoring anything after the dash)
labeling_flag     (-unlabeled ) : use the unlabeled measure instead of the default labeled one
avg_format_flag       (-micro ) : use the micro average instead of the default macro average one

Documentation: docs/index.html
That's it! you are now ready to start using TedEval.