You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Our team utilizes compute_column, and we'd like to use fixed_point literals in the ast::expression. Without fixed_point, we have to rely on floating-point, which leads to incorrect results due to the usual precision and rounding errors.
Without AST support, we have to rely on a slower fallback path.
The text was updated successfully, but these errors were encountered:
In #16088, I added support for fixed_point_scalar literals in AST here, which allows me to now pass fixed_point_scalar in an cudf::ast::operation. Although, the code compiles now, I am getting some weird behaviour when I am applying such an expression to cudf::compute_column.
So, basically, I have 2 filter expressions, each of them has a fixed_point_scalar. In one of them the value is 0.05 and in the other the value is 24. I am representing both these values as a fixed_point_scalar with a defined scale of -2 (same as my columns scale). The filter with 24 works correctly but the one with 0.05 does not. I am still trying to figure it out. But, I would like to know if anything is weird, or wrong that I am doing in this whole setup. Thanks a lot.
I wish I could use fixed_point literals in AST.
Our team utilizes compute_column, and we'd like to use fixed_point literals in the ast::expression. Without fixed_point, we have to rely on floating-point, which leads to incorrect results due to the usual precision and rounding errors.
Without AST support, we have to rely on a slower fallback path.
The text was updated successfully, but these errors were encountered: