Ontonotes data and experiments

[anoopsarkar]

Task: Convert the Ontonotes data into the CoNLL format.

The instructions for conversion are given here: http://cemantix.org/data/ontonotes.html

It also contains the script to convert to CoNLL format for all three languages: English, Chinese and Arabic.

[jerryljq]

Hi Anoop, I think this issue should also be assigned to me, right? Jiaqi.

[anoopsarkar]

@vista521 added you to the development team on github. once you accept I can add you as assignee.

[jerryljq]

@anoopsarkar Oops seems I didn't see any notifications to let me accept it. Probably there's something wrong...

[anoopsarkar]

@vista521 you should have access now.

for all the assignees, please only push to a branch for the ontonotes experiments along with your collaborators. when the convertors are written and experiments are done please send a pull request with any ontonotes conversion code and the log files for the experiments.

[liushiqi9]

@anoopsarkar
Now we have converted only English data because we don't have conll-formatted for Chinese and Arabic.

This is how it looks like for English.conll:

bn/abc/00/abc_0009   0    4    optimistic    JJ       (S(ADJP*        -    -   -   -       *   (C-ARG1*     -
bn/abc/00/abc_0009   0    5         about    IN           (PP*        -    -   -   -       *          *     -
bn/abc/00/abc_0009   0    6           the    DT        (NP(NP*        -    -   -   -       *          *     -
bn/abc/00/abc_0009   0    7        future    NN              *)   future   -   1   -       *          *     -
bn/abc/00/abc_0009   0    8            of    IN           (PP*        -    -   -   -       *          *     -
bn/abc/00/abc_0009   0    9           the    DT           (NP*        -    -   -   -       *          *   (12
bn/abc/00/abc_0009   0   10       Mideast   NNP        *)))))))       -    -   -   -    (LOC)         *)   12)
bn/abc/00/abc_0009   0   11             .     .             *))       -    -   -   -       *          *     -

bn/abc/00/abc_0009   0    0          That    DT   (TOP(SINV(SBAR(S(NP*)       -    -   -   -       *   (ARG1*     -
bn/abc/00/abc_0009   0    1            's   VBZ                   (VP*        be   -   1   -       *        *     -
bn/abc/00/abc_0009   0    2           the    DT                (NP(NP*        -    -   -   -       *        *     -
bn/abc/00/abc_0009   0    3    heartbreak    NN                      *)       -    -   -   -       *        *     -
bn/abc/00/abc_0009   0    4            of    IN                   (PP*        -    -   -   -       *        *     -
bn/abc/00/abc_0009   0    5          this    DT                   (NP*        -    -   -   -       *        *   (12
bn/abc/00/abc_0009   0    6        region    NN                 *))))))   region   -   3   -       *        *)   12)
bn/abc/00/abc_0009   0    7          says   VBZ                   (VP*)      say  01   1   -       *      (V*)    -
bn/abc/00/abc_0009   0    8           one    CD                   (NP*        -    -   -   -       *   (ARG0*     -
bn/abc/00/abc_0009   0    9         State   NNP                  (NML*        -    -   -   -   (ORG*        *     -
bn/abc/00/abc_0009   0   10    Department   NNP                      *)       -    -   -   -       *)       *     -
bn/abc/00/abc_0009   0   11      official    NN                      *)       -    -   -   -       *        *)    -
bn/abc/00/abc_0009   0   12             .     .                     *))       -    -   -   -       *        *     -

bn/abc/00/abc_0009   0    0    Whenever   WRB   (TOP(S(SBAR(WHADVP*)     -    -   -   -   *    (ARGM-TMP*)     *   (ARGM-TMP*      *             *   -
bn/abc/00/abc_0009   0    1         you   PRP                (S(NP*)     -    -   -   -   *        (ARG0*)     *            *      *             *   -
bn/abc/00/abc_0009   0    2        take   VBP                  (VP*    take  01   1   -   *           (V*)     *            *      *             *   -
bn/abc/00/abc_0009   0    3           a    DT                  (NP*      -    -   -   -   *        (ARG1*      *            *      *             *   -
bn/abc/00/abc_0009   0    4        step    NN                     *)   step   -   1   -   *             *)     *            *      *             *   -
bn/abc/00/abc_0009   0    5     forward    RB             (ADVP*))))     -    -   -   -   *    (ARGM-DIR*)     *            *)     *             *   -
bn/abc/00/abc_0009   0    6           ,     ,                     *      -    -   -   -   *             *      *            *      *             *   -
bn/abc/00/abc_0009   0    7         you   PRP                  (NP*)     -    -   -   -   *             *      *            *      *        (ARG1*)  -
bn/abc/00/abc_0009   0    8         are   VBP                  (VP*      be  03   -   -   *             *    (V*)           *      *             *   -
bn/abc/00/abc_0009   0    9       bound   VBN                  (VP*    bind  02   -   -   *             *      *          (V*)     *             *   -
bn/abc/00/abc_0009   0   10          to    TO                (S(VP*      -    -   -   -   *             *      *       (ARG1*      *             *   -
bn/abc/00/abc_0009   0   11          be    VB                  (VP*      be  03   -   -   *             *      *            *    (V*)            *   -
bn/abc/00/abc_0009   0   12      pushed   VBN                  (VP*    push  01   1   -   *             *      *            *      *           (V*)  -
bn/abc/00/abc_0009   0   13         way    RB                (ADVP*)     -    -   -   -   *             *      *            *      *    (ARGM-EXT*)  -
bn/abc/00/abc_0009   0   14        back    RB          (ADVP*)))))))     -    -   -   -   *             *      *            *)     *        (ARG2*)  -
bn/abc/00/abc_0009   0   15           .     .                    *))     -    -   -   -   *             *      *            *      *             *   -

bn/abc/00/abc_0009   0   0        Martha   NNP  (TOP(FRAG(NP*   -   -   -   -   (PERSON*   -
bn/abc/00/abc_0009   0   1       Raddatz   NNP              *)  -   -   -   -          *)  -
bn/abc/00/abc_0009   0   2             ,     ,              *   -   -   -   -          *   -
bn/abc/00/abc_0009   0   3           ABC   NNP           (NP*   -   -   -   -      (ORG*   -
bn/abc/00/abc_0009   0   4          News   NNP              *)  -   -   -   -          *)  -
bn/abc/00/abc_0009   0   5             ,     ,              *   -   -   -   -          *   -
bn/abc/00/abc_0009   0   6           the    DT           (NP*   -   -   -   -      (FAC*   -
bn/abc/00/abc_0009   0   7         State   NNP              *   -   -   -   -          *   -
bn/abc/00/abc_0009   0   8    Department   NNP              *)  -   -   -   -          *)  -
bn/abc/00/abc_0009   0   9             .     .             *))  -   -   -   -          *   -

[anoopsarkar]

According to the website: http://cemantix.org/data/ontonotes.html the script skeleton2conll.sh should work on all three Ontonotes languages. Does it do strange things when run on Chinese and Arabic?

[anoopsarkar]

Try the script on this page:
http://conll.cemantix.org/2012/data.html

[jerryljq]

I have downloaded the new script and format files and run the script based on the new files. Seems the new files worked on Chinese and Arabic.

[anoopsarkar]

Did you also repeat the conversion for English with the new script?

[jerryljq]

@anoopsarkar Yes, I think so. The new script comes with a new data set, including all three languages. I just saved all those converted files in a new folder.

[anoopsarkar]

ok. next step will be to create a new format file and config files for the new data. then the experiments can be run to train on ontonotes for each language and measure UAS on dev data.

[anoopsarkar]

English Ontonotes skel files are available at this location:

https://github.com/ontonotes/conll-formatted-ontonotes-5.0

(just for future reference)

[jerryljq]

@anoopsarkar Hi Anoop, I read the meeting notes last week. We should start doing training and testing on the dev sets. Since the data does not include the dependency tree, should we just run pos_tagger.py to train and test the POSTAG only?