Skip to content

fix: end Databricks raw string literals at the first quote - #2938

Merged
manticore-projects merged 1 commit into
JSQLParser:masterfrom
tanjiroK-coder:databricks-raw-literal-quote
Nov 18, 2026
Merged

manticore-projects merged 1 commit into
JSQLParser:masterfrom
tanjiroK-coder:databricks-raw-literal-quote

Conversation

@tanjiroK-coder

Copy link
Copy Markdown
Contributor

Under Dialect.DATABRICKS the S_CHAR_LITERAL token action lets a backslash escape the closing quote of a raw literal r'...', because the dialect enables allowBackslashEscapeCharacter, and r"..." is not lexed as a literal at all but as the identifier r followed by a backslash-escaped double-quoted string. Spark's lexer ends a raw literal at the first quote ('R\'' (~'\'')* '\'' and 'R"' (~'"')* '"' in SqlBaseLexer.g4), so a trailing backslash is plain text. As a result CCJSqlParserUtil.parse("SELECT r'a\' UNION SELECT pw FROM secrets -- '", p -> p.withDialect(Dialect.DATABRICKS)) returns one string literal and TablesNamesFinder finds no table, while Databricks runs the UNION and reads secrets; r"a\" UNION SELECT pw FROM secrets -- " behaves the same way. A table allowlist or FeaturesAllowed check run with this dialect therefore validates a different statement from the one the server executes. I found it while comparing the raw-string rules of the dialects that enable backslash escapes. Under DATABRICKS the token now ends R'...' at the first quote and lexes R"..." as a raw literal (StringValue keeps the R prefix and the double quote); other dialects split R" back into the identifier and the quoted name as before, and BigQuery raw strings are left as they are. New StringValueTest cases fail before and pass after; spotless, checkstyle, PMD, SpotBugs, the grammar ambiguity check and the test suite pass, except for 35 tests that fail on my Windows machine for environmental reasons (Mockito cannot load its MockMaker, and ParserKeywordsUtilsTest cannot create its temp file).

@manticore-projects
manticore-projects merged commit 4e24f0f into JSQLParser:master Nov 18, 2026
10 checks passed
@manticore-projects

Copy link
Copy Markdown
Contributor

Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants