Unsure about cldfbench Error

[SamPassmore]

Hi,

I have been using cldfbench to structure the kinbank database, and its subsidaries.

When updating the varikin sub-dataset (https://github.com/kinbank/varikin), I get this AssertionError from cldfbench (below).

I can't figure out what is causing the issue from the error. Do you have any suggestions of what might be causing it?

Thanks in advance,

Sam

�[32mINFO   �[0m running _cmd_makecldf on kinbank ...�[0m
�[33mWARNING�[0m forms.csv:76202:Source missing source key: Meggit�[0m
�[33mWARNING�[0m forms.csv:76203:Source missing source key: Meggit�[0m
�[33mWARNING�[0m /Users/u1123114/Documents/projects/kinbank/varikin2/kinbank/cldf/forms.csv:76450 Key `mBDC` not found in table parameters.csv�[0m
�[33mWARNING�[0m /Users/u1123114/Documents/projects/kinbank/varikin2/kinbank/cldf/forms.csv:76504 Key `mZHBW` not found in table parameters.csv�[0m
�[33mWARNING�[0m /Users/u1123114/Documents/projects/kinbank/varikin2/kinbank/cldf/forms.csv:76564 Key `fZSC` not found in table parameters.csv�[0m
Traceback (most recent call last):
  File "/Users/u1123114/Documents/projects/kinbank/varikin2/./env/bin/cldfbench", line 33, in <module>
    sys.exit(load_entry_point('cldfbench', 'console_scripts', 'cldfbench')())
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/u1123114/Documents/projects/kinbank/varikin2/env/lib/python3.12/site-packages/cldfbench/__main__.py", line 89, in main
    return args.main(args) or 0
           ^^^^^^^^^^^^^^^
  File "/Users/u1123114/Documents/projects/kinbank/varikin2/env/lib/python3.12/site-packages/pylexibank/commands/makecldf.py", line 24, in run
    with_dataset(args, 'makecldf', dataset=dataset)
  File "/Users/u1123114/Documents/projects/kinbank/varikin2/env/lib/python3.12/site-packages/cldfbench/cli_util.py", line 161, in with_dataset
    res = func(*arg, args)
          ^^^^^^^^^^^^^^^^
  File "/Users/u1123114/Documents/projects/kinbank/varikin2/env/lib/python3.12/site-packages/pylexibank/dataset.py", line 234, in _cmd_makecldf
    assert self.cldf_reader().validate(args.log)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError

[xrotwang]

The pylexibank command lexibank.makecldf runs CLDF validation right away after CLDF creation, and this fails. Unfortunately this makes debugging a bit more cumbersome. Passing the --dev flag will suppress validation. So with cldfbench lexibank.makecldf --dev ... followed by cldf validate cldf you should get better error reporting. Looking at the traceback above, it seems the first validation error is a key Meggit referenced as source but not found in the BibTeX file.

[SamPassmore]

Thanks for a fast reply Robert.

Currently I run the command:
./env/bin/python3 ./env/bin/cldfbench lexibank.makecldf --glottolog-version $(GLOTTOLOG) --concepticon-version $(CONCEPTICON) kinbank &> cldf_log.txt

But your suggesting a change to:

./env/bin/python3 ./env/bin/cldfbench lexibank.makecldf --dev --glottolog-version $(GLOTTOLOG) --concepticon-version $(CONCEPTICON) kinbank

./env/bin/python3 ./env/bin/cldfbench validate cldf &> cldf_log.txt 

Is that right?

Sam

[xrotwang]

You can also run

./env/bin/python3 ./env/bin/cldfbench validate cldf

right now. The overall command lexibank.makecldf failed just because it runs validation right away, after CLDF creation.

The line

forms.csv:76202:Source missing source key: Meggit

in your log would suggest that row 76202 in forms.csv lists Meggit as Source, but Meggit isn't a valid BibTeX key.

[xrotwang]

As far as I can tell, there's only an entry Meggitt_1964 here https://github.com/kinbank/varikin/blob/9791f96a3ea18c21a262eac7e61a9315e2da4130/kinbank/raw/sources.bib#L17628

[xrotwang]

Btw.: I can't reproduce the issue with current HEAD of https://github.com/kinbank/varikin .

[SamPassmore]

Hi Robert,

Apologies for the long delay in getting back to this.
Has the validate command been deprecated in cldfbench?

Sam

[xrotwang]

Apologies for the long delay in getting back to this. Has the validate command been deprecated in cldfbench?

I don't know why I suggested cldfbench validate above, tbh. validate has always been a subcommand of the cldf program, not of cldfbench. And cldf validate is not deprecated.