USAGE

Most parameters are specified in a config file:

 itg -configfile <configfile>

Specific paramters can be overridden on the command line

 itg -configfile <configfile> -lm_weight 0.3

See README.params for complete list of parameters.
Example config files can be found in the Test directory.
Test/999999.config provides an example Chinese-English setup.
If no -configfile option is given, the default is to look for 
a file named "config" in the current directory.


FILE FORMATS 


Decoder Input Sentences

For decoding, the input sentences are in the file specified by the
"l2corpus" parameter, one line per setences, with words converted to
numeric IDs.


Grammar / Translation Table

There are two file formats for the grammar, one for simple ITG, where 
all phrase pairs are generated by a single preterminal, and one for
generalized ITG, with many preterminals and nonterminals.

Simple ITG File Format:
There are two files, one file for the grammar over nonterminals, and one
for the phrase table.  The nonterminal grammar file has the following format
(see /u/post/code/itgparse/ebtg.SABC):

 one line containing all nonterminals in the grammar
 one line containing the preterminals
 one line per rule

The start symbol for the grammar is hardcoded as "S", so you must use "S" 
in your rules.  In the rules, -> means straight and -< means inverted.
The format for rules is 

 lhs -> rhs prob

The phrase table for simple ITG format is one line per phrase pair.  All
phrase pairs are produced by the single preterminal specified on the 
second line of the grammar file.  The format of each line is:

 english_phrase chinese_phrase prob

where both the English and Chinese phrases are sequences of numeric 
word IDs terminated by zero.


Generalized ITG Format: 

If the grammar file contains no rules (ony the list of nonterminals),
the phrase table is assumed to be in generalized ITG format.  The format
is one rule per line:

 probability ruletype lhs chinese_rhs english_rhs

The lefthand side is a numeric nonterminal ID.  The ruletype 4
indicates an inverted rule.  The righthand side consists of a chinese
righthand side and an engligh righthand side, both terminated with
zero.  Within the lhs and rhs, negative numbers indicate nonterminal
IDs, and positive numbers indicate word (terminal) IDs.  The english
righthand side can contain a mixture of nonterminals and terminals, 
but the chinese righthand must contain only nonterminals or only 
terminals.

rule types:
   type=0: rhs has Chinese terminals only
   type=1: rhs is a translation pair (without non-terminals)
     NOTE: currently, type 1 has been merged to type 0
   type=2: similar to type=1, but with 1 non-terminal
   type=3: similar to type=2, but with 2 non-terminals
     NOTE: currently, type 3 has been merged to type 2
   type=4: similar to type=3, but the 2 non-terminals are reversed

Because of the merge, we have
  type 0 for translation pairs (only terminals)
  type 2 for monotonic translation (w/ 1 or 2 nonterminals)
  type 4 for non-monotonic translation (w/ 1 or 2 nonterminals)

Language Model

The ngram language model is in ARPA format, produced by SRILM, with words
represented by numeric IDs.
