You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Describe the bug
I used the following command over one of the 2022 Wikidata JSON dumps and I didn't have enough free space on my disk for the output *.tsv files
The process has been unsuccessful and the /tmp have been completely occupied with kgtk-graph-cache-sh200.sqlite3.db (about 63 GB). The SQLite file seems to remain after some other successful importing as well.
To Reproduce
Not sure how to reproduce the situation, but I think the problem was due to a lack of free space.
Expected behavior
The /tmp is better to be cleared after either successful or failed importing process
The text was updated successfully, but these errors were encountered:
Ahh, I have another command that runs after the import. That command is : kgtk query -i edgefile.tsv --match '(n1)-[:P31]->(class), (n1)-[p]->(n2)' --where 'class IN ["Q11173","Q12136","Q7187","Q8054"]' --return 'n1, p, n2' > ./kgtk_output.tsv
Working with large data files requires some care. As Craig suggested, make sure the edge file that was produced is compressed to not waste any space.
For Kypher, use the --gc option to direct it to use a graph cache file in a location that has enough available space. For example: kgtk query --gc /data/wikidata.sqlite3.db ....
See https://github.com/usc-isi-i2/kgtk/blob/dev/docs/transform/query.md#graph-cache
Describe the bug
I used the following command over one of the 2022 Wikidata JSON dumps and I didn't have enough free space on my disk for the output *.tsv files
The process has been unsuccessful and the /tmp have been completely occupied with
kgtk-graph-cache-sh200.sqlite3.db
(about 63 GB). The SQLite file seems to remain after some other successful importing as well.To Reproduce
Not sure how to reproduce the situation, but I think the problem was due to a lack of free space.
Expected behavior
The /tmp is better to be cleared after either successful or failed importing process
The text was updated successfully, but these errors were encountered: