feat(snowflake)!: support transpilation of BITSHIFTLEFT and BITSHIFTRIGHT

[fivetran-kwoodbeck]

DuckDB has support for left and right shift (<< and >>) up to INT128, but by default it assumes INT32 and needs casting in order to prevent it from throwing errors in various situations. See the Jira tickets for full testing.

Snowflake claims it always returns an INT128 (for INT input). In practice, it scales based on result and ranges from NUMBER(3,0) to NUMBER(38,0) depending on what number is returned. Snowflake also supports BINARY input.

Snowflake does not throw errors when you shift over the max int, DuckDB does.

Documentation:
https://docs.snowflake.com/en/sql-reference/functions/bitshiftleft
https://docs.snowflake.com/en/sql-reference/functions/bitshiftright

[github-actions]

SQLGlot Integration Test Results

Comparing:

• this branch (sqlglot:feature/transpile-bitshift, sqlglot version: feature/transpile-bitshift)
• baseline (main, sqlglot version: 28.5.1.dev43)

⚠️ Limited to dialects: duckdb, snowflake

By Dialect

dialect	main	sqlglot:feature/transpile-bitshift	difference	links
duckdb -> duckdb	4003/4003 passed (100.0%)	4003/4003 passed (100.0%)	No change	full result / delta
snowflake -> duckdb	626/1085 passed (57.7%)	626/1085 passed (57.7%)	No change	full result / delta
snowflake -> snowflake	981/1085 passed (90.4%)	981/1085 passed (90.4%)	No change	full result / delta

Overall

main: 6173 total, 5610 passed (pass rate: 90.9%), sqlglot version: 28.5.1.dev43

sqlglot:feature/transpile-bitshift: 6173 total, 5610 passed (pass rate: 90.9%), sqlglot version: feature/transpile-bitshift

Difference: No change

[georgesittas]

This needs testing, but I'm skeptical about the approach. May need to sync in Slack about this.

[fivetran-kwoodbeck]

This needs testing, but I'm skeptical about the approach. May need to sync in Slack about this.

@georgesittas Please see the Jira ticket for all tests that were run prior to the first push. There's about 300 test queries against Snowflake, transpiled, run against DuckDB and results checked against one another.

[georgesittas]

We need to add tests.

[georgesittas]

@fivetran-kwoodbeck @VaggelisD take a look at the refactored PR when you get the chance, took a stab at cleaning it up a bit.

[VaggelisD]

Only got one question, @fivetran-kwoodbeck if the refactored logic & tests check out feel free to merge

[VaggelisD]

This generates a BINARY in Snowflake but an INT in DuckDB, this is intentional given that we don't annotate the types (?)

sf> SELECT system$typeof(BITSHIFTRIGHT(X'FF', 4)), BITSHIFTRIGHT(X'FF', 4);
BINARY[LOB] | 0F
-- | --


duckdb> SELECT CAST(255 AS INT128) >> 4;
┌─────────────────────────────┐
│ (CAST(255 AS HUGEINT) >> 4) │
│           int128            │
├─────────────────────────────┤
│             15              │
└─────────────────────────────┘

[georgesittas]

What if we annotate the input w/ snowflake? I think preserving the type requires type inference to run first.

[fivetran-kwoodbeck]

The latest patch, which also fixes the fundamental HEX transpilation issue, resolves this. I've made HEX resolve to binary by default.

SELECT CAST(CAST(UNHEX('FF') AS BIT) >> 4 AS BLOB);

[fivetran-kwoodbeck]

Hi @VaggelisD The tests don't pass and I'm not sure what's the best way to deal with it (a lot of back and forth). There are 2 issues, both of which can be seen here:

SELECT BITSHIFTLEFT(X'0080'::BINARY, 1);
SELECT BITSHIFTRIGHT(BITSHIFTLEFT(X'0080'::BINARY, 1), 1);

The first is INT to BLOB issue, when transpiled:

SELECT CAST(CAST(CAST(128 AS BLOB) AS BIT) << 1 AS BLOB);

Conversion Error: Unimplemented type for cast (INTEGER -> BLOB)
LINE 1: SELECT CAST(CAST(CAST(128 AS BLOB) AS BIT) << 1 AS BLOB)

I think it needs an UNHEX and cast to BIT (see below). The second, if we fix the hex issue, then has another problem where the first parameter is not seen as BINARY so it assumes INT, which is wrong:

SELECT CAST(CAST(CAST(UNHEX('0080') AS BIT) << 1 AS BLOB) AS INT128) >> 1

Conversion Error: Unimplemented type for cast (BLOB -> HUGEINT) LINE 1: SELECT CAST(CAST(CAST(UNHEX('0080') AS BIT) << 1 AS BLOB) AS INT128... ^

Which I think can be solved with an extra annotate_types called on the first parameter, but I'm not entirely sure.

[fivetran-kwoodbeck]

This line is invalid, it adds a TO_HEX to the previous FROM_HEX, which turns it into a string that is then cast to BINARY.

[fivetran-kwoodbeck]

This returns a different result in DuckDB than 'CC'