i get an error when generating wordwise and xray

[daenghunter]

Checkboxes

• I have read the installation and usage documents.
• I have not found similar open/closed issue or disscussion.
• Reboot doesn't fix the problem.

Describe the bug and copy the error message

calibre, version 7.12.0 (win32, embedded-python: True)
Tonnerre de Brest!: An error occurred, please copy error message then report bug at GitHub.

Starting job: Generating Word Wise and X-Ray for A Psalm for the Wild-Built
Job: "Generating Word Wise and X-Ray for A Psalm for the Wild-Built" failed with error:
Traceback (most recent call last):
File "calibre\gui2\threaded_jobs.py", line 85, in start_work
File "calibre_plugins.worddumb.parse_job", line 211, in do_job
File "calibre_plugins.worddumb.utils", line 53, in run_subprocess
File "subprocess.py", line 571, in run
subprocess.CalledProcessError: Command '['py', 'C:\Users\OWNER\AppData\Roaming\calibre\plugins\WordDumb.zip', '{"book_id": 3, "book_path": "C:\\Users\\OWNER\\OneDrive\\Calibre\\Becky Chambers\\A Psalm for the Wild-Built (3)\\A Psalm for the Wild-Built - Becky Chambers.kfx", "mi": null, "book_fmt": "KFX", "book_lang": "en", "useragent": "WordDumb/3.32.0 (https://github.com/xxyzz/WordDumb)", "plugin_path": "C:\\Users\\OWNER\\AppData\\Roaming\\calibre\\plugins\\WordDumb.zip", "spacy_model": "en_core_web_md", "create_ww": true, "create_x": true, "asin": "B08H831J18", "acr": "CR!YPSVDULJ5J3O5INDW2BHCZ8GEBKN", "revision": "", "kfx_json": null, "mobi_html": null, "mobi_codec": ""}', '{"use_pos": true, "search_people": false, "model_size": "md", "zh_wiki_variant": "cn", "mediawiki_api": "", "add_locator_map": false, "preferred_formats": ["KFX", "AZW3", "AZW", "MOBI", "EPUB"], "use_all_formats": false, "minimal_x_ray_count": 1, "choose_format_manually": false, "wiktionary_gloss_lang": "en", "kindle_gloss_lang": "en", "use_gpu": false, "cuda": "cu121", "use_wiktionary_for_kindle": false, "remove_link_styles": false, "python_path": "", "show_change_kindle_ww_lang_warning": true, "ca_wiktionary_difficulty_limit": 5, "cs_wiktionary_difficulty_limit": 5, "da_wiktionary_difficulty_limit": 5, "de_wiktionary_difficulty_limit": 5, "el_wiktionary_difficulty_limit": 5, "en_wiktionary_difficulty_limit": 5, "es_wiktionary_difficulty_limit": 5, "fi_wiktionary_difficulty_limit": 5, "fr_wiktionary_difficulty_limit": 5, "he_wiktionary_difficulty_limit": 5, "hr_wiktionary_difficulty_limit": 5, "it_wiktionary_difficulty_limit": 5, "ja_wiktionary_difficulty_limit": 5, "ko_wiktionary_difficulty_limit": 5, "lt_wiktionary_difficulty_limit": 5, "mk_wiktionary_difficulty_limit": 5, "nl_wiktionary_difficulty_limit": 5, "no_wiktionary_difficulty_limit": 5, "pl_wiktionary_difficulty_limit": 5, "pt_wiktionary_difficulty_limit": 5, "ro_wiktionary_difficulty_limit": 5, "ru_wiktionary_difficulty_limit": 5, "sl_wiktionary_difficulty_limit": 5, "sv_wiktionary_difficulty_limit": 5, "uk_wiktionary_difficulty_limit": 5, "zh_wiktionary_difficulty_limit": 5}']' returned non-zero exit status 1.

Called with args: (ParseJobData(book_id=3, book_path='C:\Users\OWNER\OneDrive\Calibre\Becky Chambers\A Psalm for the Wild-Built (3)\A Psalm for the Wild-Built - Becky Chambers.kfx', mi=<calibre.ebooks.metadata.book.base.Metadata object at 0x000001AC05D2C610>, book_fmt='KFX', book_lang='en', useragent='WordDumb/3.32.0 (https://github.com/xxyzz/WordDumb)', plugin_path='C:\Users\OWNER\AppData\Roaming\calibre\plugins\WordDumb.zip', spacy_model='en_core_web_md', create_ww=True, create_x=True, asin='B08H831J18', acr='CR!YPSVDULJ5J3O5INDW2BHCZ8GEBKN', revision='', kfx_json=None, mobi_html=None, mobi_codec=''),) {'notifications': <queue.Queue object at 0x000001AC09001A10>, 'abort': <threading.Event at 0x1ac09003890: unset>, 'log': <calibre.utils.logging.GUILog object at 0x000001AC09003390>}
Traceback (most recent call last):
File "", line 198, in run_module_as_main
File "", line 88, in run_code
File "C:\Users\OWNER\AppData\Roaming\calibre\plugins\WordDumb.zip_main.py", line 37, in
File "C:\Users\OWNER\AppData\Roaming\calibre\plugins\WordDumb.zip\parse_job.py", line 237, in create_files
File "C:\Users\OWNER\AppData\Roaming\calibre\plugins\WordDumb.zip\parse_job.py", line 763, in load_spacy
File "C:\Users\OWNER\AppData\Roaming\calibre\plugins\worddumb-libs-py3.12\spacy_init.py", line 6, in
from .errors import setup_default_warnings
File "C:\Users\OWNER\AppData\Roaming\calibre\plugins\worddumb-libs-py3.12\spacy\errors.py", line 3, in
from .compat import Literal
File "C:\Users\OWNER\AppData\Roaming\calibre\plugins\worddumb-libs-py3.12\spacy\compat.py", line 39, in
from thinc.api import Optimizer # noqa: F401
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\OWNER\AppData\Roaming\calibre\plugins\worddumb-libs-py3.12\thinc\api.py", line 1, in
from .backends import (
File "C:\Users\OWNER\AppData\Roaming\calibre\plugins\worddumb-libs-py3.12\thinc\backends_init_.py", line 17, in
from .cupy_ops import CupyOps
File "C:\Users\OWNER\AppData\Roaming\calibre\plugins\worddumb-libs-py3.12\thinc\backends\cupy_ops.py", line 16, in
from .numpy_ops import NumpyOps
File "thinc\backends\numpy_ops.pyx", line 1, in init thinc.backends.numpy_ops
ValueError: numpy.dtype size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject

Plugin settings and reproduce steps

so whenever i try to get the files for kfx files. it dpesnt work and i get the same error message, it happens in both of my laptops

even reinstalling and deleting all the plugins doesnt solve the problem

Generated files, screenshots or videos

[xxyzz]

Please delete "C:\Users\OWNER\AppData\Roaming\calibre\plugins\worddumb-libs-py3.12", some dependency packages probably are outdated.

[xxyzz]

New numpy 2.0 release breaks spaCy model... Downgrade numpy should fix the error.

[daenghunter]

i downgraded it but i still get an error

Starting job: Generating Word Wise and X-Ray for The Midnight Library
Job: "Generating Word Wise and X-Ray for The Midnight Library" failed with error:
Traceback (most recent call last):
File "calibre\gui2\threaded_jobs.py", line 85, in start_work
File "calibre_plugins.worddumb.parse_job", line 211, in do_job
File "calibre_plugins.worddumb.utils", line 53, in run_subprocess
File "subprocess.py", line 571, in run
subprocess.CalledProcessError: Command '['py', 'C:\Users\OWNER\AppData\Roaming\calibre\plugins\WordDumb.zip', '{"book_id": 982, "book_path": "I:\\Calibre\\Matt. Haig\\The Midnight Library (982)\\The Midnight Library - Matt. Haig.kfx", "mi": null, "book_fmt": "KFX", "book_lang": "en", "useragent": "WordDumb/3.32.0 (https://github.com/xxyzz/WordDumb)", "plugin_path": "C:\\Users\\OWNER\\AppData\\Roaming\\calibre\\plugins\\WordDumb.zip", "spacy_model": "en_core_web_md", "create_ww": true, "create_x": true, "asin": "BB92627JZS", "acr": "CR!D4IWF42E8MMGJAX9GJCLKCZCTK5V", "revision": "", "kfx_json": null, "mobi_html": null, "mobi_codec": ""}', '{"use_pos": true, "search_people": true, "model_size": "md", "zh_wiki_variant": "cn", "mediawiki_api": "", "add_locator_map": false, "preferred_formats": ["KFX", "AZW3", "AZW", "MOBI", "EPUB"], "use_all_formats": false, "minimal_x_ray_count": 1, "choose_format_manually": false, "wiktionary_gloss_lang": "en", "kindle_gloss_lang": "en", "use_gpu": false, "cuda": "cu121", "use_wiktionary_for_kindle": false, "remove_link_styles": false, "python_path": "", "show_change_kindle_ww_lang_warning": true, "ca_wiktionary_difficulty_limit": 5, "cs_wiktionary_difficulty_limit": 5, "da_wiktionary_difficulty_limit": 5, "de_wiktionary_difficulty_limit": 5, "el_wiktionary_difficulty_limit": 5, "en_wiktionary_difficulty_limit": 5, "es_wiktionary_difficulty_limit": 5, "fi_wiktionary_difficulty_limit": 5, "fr_wiktionary_difficulty_limit": 5, "he_wiktionary_difficulty_limit": 5, "hr_wiktionary_difficulty_limit": 5, "it_wiktionary_difficulty_limit": 5, "ja_wiktionary_difficulty_limit": 5, "ko_wiktionary_difficulty_limit": 5, "lt_wiktionary_difficulty_limit": 5, "mk_wiktionary_difficulty_limit": 5, "nl_wiktionary_difficulty_limit": 5, "no_wiktionary_difficulty_limit": 5, "pl_wiktionary_difficulty_limit": 5, "pt_wiktionary_difficulty_limit": 5, "ro_wiktionary_difficulty_limit": 5, "ru_wiktionary_difficulty_limit": 5, "sl_wiktionary_difficulty_limit": 5, "sv_wiktionary_difficulty_limit": 5, "uk_wiktionary_difficulty_limit": 5, "zh_wiktionary_difficulty_limit": 5}']' returned non-zero exit status 1.

Called with args: (ParseJobData(book_id=982, book_path='I:\Calibre\Matt. Haig\The Midnight Library (982)\The Midnight Library - Matt. Haig.kfx', mi=<calibre.ebooks.metadata.book.base.Metadata object at 0x0000018CC97B3190>, book_fmt='KFX', book_lang='en', useragent='WordDumb/3.32.0 (https://github.com/xxyzz/WordDumb)', plugin_path='C:\Users\OWNER\AppData\Roaming\calibre\plugins\WordDumb.zip', spacy_model='en_core_web_md', create_ww=True, create_x=True, asin='BB92627JZS', acr='CR!D4IWF42E8MMGJAX9GJCLKCZCTK5V', revision='', kfx_json=None, mobi_html=None, mobi_codec=''),) {'notifications': <queue.Queue object at 0x0000018CD0FC7E50>, 'abort': <threading.Event at 0x18cc97b1e50: unset>, 'log': <calibre.utils.logging.GUILog object at 0x0000018CC97B1710>}
Traceback (most recent call last):
File "", line 198, in run_module_as_main
File "", line 88, in run_code
File "C:\Users\OWNER\AppData\Roaming\calibre\plugins\WordDumb.zip_main.py", line 37, in
File "C:\Users\OWNER\AppData\Roaming\calibre\plugins\WordDumb.zip\parse_job.py", line 237, in create_files
File "C:\Users\OWNER\AppData\Roaming\calibre\plugins\WordDumb.zip\parse_job.py", line 763, in load_spacy
File "C:\Users\OWNER\AppData\Roaming\calibre\plugins\worddumb-libs-py3.12\spacy_init.py", line 6, in
from .errors import setup_default_warnings
File "C:\Users\OWNER\AppData\Roaming\calibre\plugins\worddumb-libs-py3.12\spacy\errors.py", line 3, in
from .compat import Literal
File "C:\Users\OWNER\AppData\Roaming\calibre\plugins\worddumb-libs-py3.12\spacy\compat.py", line 39, in
from thinc.api import Optimizer # noqa: F401
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\OWNER\AppData\Roaming\calibre\plugins\worddumb-libs-py3.12\thinc\api.py", line 1, in
from .backends import (
File "C:\Users\OWNER\AppData\Roaming\calibre\plugins\worddumb-libs-py3.12\thinc\backends_init_.py", line 17, in
from .cupy_ops import CupyOps
File "C:\Users\OWNER\AppData\Roaming\calibre\plugins\worddumb-libs-py3.12\thinc\backends\cupy_ops.py", line 16, in
from .numpy_ops import NumpyOps
File "thinc\backends\numpy_ops.pyx", line 1, in init thinc.backends.numpy_ops
ValueError: numpy.dtype size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject

[hosford42]

I'm getting this too. Freshly installed SpaCy, with no other packages installed. I don't use your package at all, @xxyzz , so I'm thinking this is a SpaCy issue.

[hosford42]

Confirmed:

explosion/spaCy#13528

[xxyzz]

This plugin downloads its dependencies in calibre's plugin folder and doesn't use packages installed for the system. A temporary fix is downgrade numpy in that folder manually:

py -m pip install -U -t C:\Users\OWNER\AppData\Roaming\calibre\plugins\worddumb-libs-py3.12 numpy==1.26.4

Please notice the install path is for Python 3.12, it needs to be changed for other Python versions.

[woofingcough]

This plugin downloads its dependencies in calibre's plugin folder and doesn't use packages installed for the system. A temporary fix is downgrade numpy in that folder manually:

py -m pip install -U -t C:\Users\OWNER\AppData\Roaming\calibre\plugins\worddumb-libs-py3.12 numpy==1.16.4

Could you give me a command for MacOS as well?

[xxyzz]

Could you give me a command for MacOS as well?

python -m pip install -U -t  ~/Library/Preferences/calibre/plugins/worddumb-libs-py3.12 numpy==1.26.4

[woofingcough]

Could you give me a command for MacOS as well?

python -m pip install -U -t  ~/Library/Preferences/calibre/plugins/worddumb-libs-py3.12 numpy==1.16.4

Much appreciated!

[woofingcough]

Could you give me a command for MacOS as well?

python -m pip install -U -t  ~/Library/Preferences/calibre/plugins/worddumb-libs-py3.12 numpy==1.16.4

By the way, the problem persists even though I have applied this… and 1.16.4 has been deprecated, so I used 1.26.4.

[u0398]

This plugin downloads its dependencies in calibre's plugin folder and doesn't use packages installed for the system. A temporary fix is downgrade numpy in that folder manually:

py -m pip install -U -t C:\Users\OWNER\AppData\Roaming\calibre\plugins\worddumb-libs-py3.12 numpy==1.16.4

I could not get 1.16.4 to install (ERROR: Failed building wheel for numpy), but 1.26.4 installed and WordDumb is working.

[xxyzz]

Sorry for the typo🤦 Should downgrade to 1.26.4

[woofingcough]

Could you give me a command for MacOS as well?

python -m pip install -U -t  ~/Library/Preferences/calibre/plugins/worddumb-libs-py3.12 numpy==1.16.4

By the way, the problem persists even though I have applied this… and 1.16.4 has been deprecated, so I used 1.26.4.

Deleting the plugin and the remaining files and reinstalling the plugin again before typing in this solved the problem!