You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Debugging the code (at least from python) i noticed that the table created with delta 0.10 has "timestamp" as a primary type, while new tables now have "timestamp_ntz"
If i add the timezone (for example UTC), even if the table is created with delta 0.10, the write is successful
The text was updated successfully, but these errors were encountered:
I am additionally have the same issue when writing new partitions with type timestamp in 0.18.1, when time stamps are structured as for example: '2015-10-30T06:40:15.000Z', year-month-dayThour-min-month.000Z
We fixed a longstanding bug where timestamps where incorrect, this has now been correct and was a backwards incomatible change in some areas, additionally the pyarrow engine however incorrectly writes UTC timestamps as Z, this is something we cannot configure in pyarrow
Is there a way to migrate the schema from "timestamp" to "timestamp_ntz" without recreating the table? (and load all the historical data)
I tried with schema_mode: overwrite but i get the same error. It seems that delta is unable to write the new "timestamp_ntz" type over the legacy "timestamp".
Environment
Delta-rs version:
Binding: 0.18.0
Environment:
Bug
What happened:
I have a table written with
delta-rs 0.10.0
. The schema is:I'm triyng to write a new partition on that table with the following schema:
But i get:
DeltaError: Generic DeltaTable error: Writer features must be specified for writerversion >= 7, please specify: TimestampWithoutTimezone
.With deltalake 0.16.2 worked fine. Now i dumped to 0.18.0 and i get this error with tables created with an old delta-rs client.
If i the table is created with newer delta-rs client, this doesn't happen.
How to reproduce it:
deltalake==0.10.0
deltalake==0.18.0
More details:
The text was updated successfully, but these errors were encountered: