Skip to content

[AIR] CNXT-2688: fail on scheme-less /Volumes/ paths in fs commands - #7862

Open
vinchenzo-db wants to merge 10 commits into
mainfrom
vinchenzo-db/cnxt-2688
Open

vinchenzo-db wants to merge 10 commits into
mainfrom
vinchenzo-db/cnxt-2688

Conversation

@vinchenzo-db

@vinchenzo-db vinchenzo-db commented Nov 21, 2026 •

Copy link
Copy Markdown
Contributor

Changes

databricks fs commands now warn on stderr, once per command and not during shell completion, when a path argument starts with /Volumes/ or /Skills/ but has no dbfs: scheme, and keep treating it as a local path:

Warn: Paths without a scheme are local paths: /Volumes/main/default/vol. For a Unity Catalog volume or skill, add the dbfs: scheme

databricks fs cp --help now shows a UC Volume example (dbfs:/Volumes/my_catalog/my_schema/my_volume/data.csv) and says that a scheme-less path is local even if it starts with /Volumes/.

No behavior change otherwise: scheme-less paths stay local paths, and /Volumes/... is not auto-routed to the Files API.

Why

Fixes CNXT-2688 (AIR CUJ PE64 papercut). databricks fs cp file /Volumes/main/<schema>/<vol>/ and fs mkdirs /Volumes/... were treated as local paths: they failed with a local "permission denied" on the laptop, or silently succeeded in the wrong place where /Volumes is writable. Only dbfs:/Volumes/... reached Unity Catalog, and fs --help did not say so.

Supervisor's call: warn, don't reject, because /Volumes is the macOS mount point for local drives, so a scheme-less /Volumes/... path can be a legitimate local path. The warning removes the silent part of the papercut without breaking those paths or adding a new scheme.

Tests

  • New acceptance test acceptance/cmd/fs/uc-path-without-scheme pins the warning and the local behavior for /Volumes/ and /Skills/ paths, one warning for a cp with two such arguments, and no warning during completion. It runs on Windows with MSYS_NO_PATHCONV=1, as acceptance/cmd/api/* do.
  • Unit test TestWarnSchemelessUCPaths in cmd/fs/helpers_test.go covers the warning and the paths that must not warn.
  • go test ./cmd/fs/, go test ./acceptance -run TestAccept/cmd/fs/ and task lint-q pass locally.

This pull request and its description were written by Isaac.

A path such as /Volumes/main/default/vol/data.csv without the dbfs:
scheme was treated as a local path, so `databricks fs cp` and
`fs mkdirs` silently acted on the laptop. Fail with a hint to use
dbfs:/Volumes/... instead, and show a UC Volume example in `fs cp --help`.

Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
@vinchenzo-db

Copy link
Copy Markdown
Contributor Author

/review

@review-oss-bot review-oss-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review at 907356b. Each finding was verified against the code.

Not ready yet: 1 blocking issue(s), tagged [blocking].

This PR adds a guard in filerForPath that rejects scheme-less /Volumes/ paths with a helpful error pointing to dbfs:/Volumes/…, and updates the cp --help text. The main issue is that both the help text (cp.go:217) and the error message (helpers.go:42) say /Volumes/… "is a local path" while the code now refuses to treat it as one—the documentation and the implementation disagree. The new guard also omits /Skills/, which has the same silent-wrong-filesystem risk, and the testing rule asks for an acceptance test rather than only a unit test for this user-visible error.

Comment thread cmd/fs/cp.go Resolved
Comment thread cmd/fs/helpers.go Resolved
Comment thread cmd/fs/helpers_test.go Resolved
Co-authored-by: Isaac <no-reply@databricks.com>
@vinchenzo-db

Copy link
Copy Markdown
Contributor Author

/review

@github-actions

github-actions Bot commented Nov 21, 2026 •

Copy link
Copy Markdown
Contributor

Commit: 1bb23ae21 | Workflow run

Regression test report

Selected 1 changed tests (limit=20, 0 not selected)

1 tests: regression=1 unreleased=0 coverage=0 failing=0
test branch main (a4388d7) latest (v1.27.0)
TestAccept/cmd/fs/uc-path-without-scheme ✅ ❌ ❌
TestAccept/cmd/fs/uc-path-without-scheme: branch ✅ | main (a4388d7) ❌ | latest (v1.27.0) ❌

main (a4388d7):

acceptance_test.go:1402: 
        	Error Trace:	/home/runner/work/cli/cli/libs/testdiff/testdiff.go:22
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1402
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1143
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:686
        	Error:      	Not equal: 
        	            	expected: "\n>>> musterr [CLI] fs ls /Volumes/main/default/vol\nWarn: Paths without a scheme are local paths: /Volumes/main/default/vol. For a Unity Catalog volume or skill, add the dbfs: scheme\nError: no such directory: /Volumes/main/default/vol\n\n>>> musterr [CLI] fs ls /Skills/foo/bar\nWarn: Paths without a scheme are local paths: /Skills/foo/bar. For a Unity Catalog volume or skill, add the dbfs: scheme\nError: no such directory: /Skills/foo/bar\n\n>>> musterr [CLI] fs cp /Volumes/main/default/vol/a.csv /Volumes/main/default/vol/b.csv\nWarn: Paths without a scheme are local paths: /Volumes/main/default/vol/a.csv, /Volumes/main/default/vol/b.csv. For a Unity Catalog volume or skill, add the dbfs: scheme\nError: file does not exist: /Volumes/main/default/vol/a.csv\n\n>>> [CLI] __complete fs ls /Volumes/main/default/vol/\n:1\nCompletion ended with directive: ShellCompDirectiveError\n"
        	            	actual  : "\n>>> musterr [CLI] fs ls /Volumes/main/default/vol\nError: no such directory: /Volumes/main/default/vol\n\n>>> musterr [CLI] fs ls /Skills/foo/bar\nError: no such directory: /Skills/foo/bar\n\n>>> musterr [CLI] fs cp /Volumes/main/default/vol/a.csv /Volumes/main/default/vol/b.csv\nError: file does not exist: /Volumes/main/default/vol/a.csv\n\n>>> [CLI] __complete fs ls /Volumes/main/default/vol/\n:1\nCompletion ended with directive: ShellCompDirectiveError\n"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -2,3 +2,2 @@
        	            	 >>> musterr [CLI] fs ls /Volumes/main/default/vol
        	            	-Warn: Paths without a scheme are local paths: /Volumes/main/default/vol. For a Unity Catalog volume or skill, add the dbfs: scheme
        	            	 Error: no such directory: /Volumes/main/default/vol
        	            	@@ -6,3 +5,2 @@
        	            	 >>> musterr [CLI] fs ls /Skills/foo/bar
        	            	-Warn: Paths without a scheme are local paths: /Skills/foo/bar. For a Unity Catalog volume or skill, add the dbfs: scheme
        	            	 Error: no such directory: /Skills/foo/bar
        	            	@@ -10,3 +8,2 @@
        	            	 >>> musterr [CLI] fs cp /Volumes/main/default/vol/a.csv /Volumes/main/default/vol/b.csv
        	            	-Warn: Paths without a scheme are local paths: /Volumes/main/default/vol/a.csv, /Volumes/main/default/vol/b.csv. For a Unity Catalog volume or skill, add the dbfs: scheme
        	            	 Error: file does not exist: /Volumes/main/default/vol/a.csv
        	Test:       	TestAccept/cmd/fs/uc-path-without-scheme
        	Messages:   	cmd/fs/uc-path-without-scheme/output.txt vs /tmp/TestAcceptcmdfsuc-path-without-scheme765078666/002/output.txt
FAIL

latest (v1.27.0): same output as main (a4388d7)

@review-oss-bot review-oss-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review at 73153e3. Each finding was verified against the code.

Not ready yet: 1 blocking issue(s), tagged [blocking].

This PR adds a guard in filerForPath that rejects scheme-less /Volumes/ and /Skills/ paths with a helpful hint to add dbfs:. It also updates fs cp --help and adds an acceptance test plus a unit test. The guard catches a real UX papercut, but it also blocks legitimate local /Volumes/ paths on macOS (the OS mount point) with no workaround because the file: scheme is rejected too. The new acceptance test skips Windows entirely instead of using the documented MSYS_NO_PATHCONV workaround.

Comment thread cmd/fs/helpers.go Resolved
@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Nov 21, 2026 •

Copy link
Copy Markdown
Collaborator

✅ Integration test report

Commit: 1bb23ae

Run: 44875866862

Env 💚​RECOVERED ✅​pass 🙈​skip Time
💚​ aws linux-2core-8gb 1 272 20 4:43
💚​ aws-windows-latest-4core-16gb 1 274 18 4:11
💚​ azure linux-2core-8gb 1 271 20 4:01
💚​ azure-windows-latest-4core-16gb 1 273 18 4:04
💚​ gcp linux-2core-8gb 1 272 20 4:08
💚​ gcp-windows-latest-4core-16gb 1 274 18 4:11
Test Name aws linux-2core-8gb aws-windows-latest-4core-16gb azure linux-2core-8gb azure-windows-latest-4core-16gb gcp linux-2core-8gb gcp-windows-latest-4core-16gb
💚​ TestAccept 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
Top 3 slowest tests (at least 2 minutes):
duration env testname
4:10 aws-windows-latest-4core-16gb TestAccept
4:09 gcp-windows-latest-4core-16gb TestAccept
4:02 azure-windows-latest-4core-16gb TestAccept

On macOS /Volumes/ is the mount point for local drives, so rejecting
every scheme-less /Volumes/ path would block real local paths. Keep
them local and print a warning on stderr instead. Run the acceptance
test on Windows with MSYS_NO_PATHCONV.

Co-authored-by: Isaac <no-reply@databricks.com>
@vinchenzo-db

Copy link
Copy Markdown
Contributor Author

/review

@review-oss-bot review-oss-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review at 6fd3053. Each finding was verified against the code.

No blocking issues found. 2 non-blocking comment(s) below.

This PR adds a stderr warning when databricks fs commands receive a /Volumes/ or /Skills/ path without the dbfs: scheme, and updates cp --help with a UC Volume example and a note that scheme-less paths are local. The core logic is clean and the acceptance test pins the warning, but the unit test doesn't verify the new warning and the help-text clarification only covers cp.

Comment thread cmd/fs/helpers_test.go Resolved
Comment thread cmd/fs/cp.go Resolved
…bcommands

Co-authored-by: Isaac <no-reply@databricks.com>
@vinchenzo-db

Copy link
Copy Markdown
Contributor Author

/review

@review-oss-bot review-oss-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review at a57c267. Each finding was verified against the code.

No blocking issues found. 1 non-blocking comment(s) below.

This PR adds a stderr warning when databricks fs commands receive a /Volumes/ or /Skills/ path without the dbfs: scheme, and enriches the --help text of every subcommand to clarify that a scheme-less path is always local. The warning logic and tests look correct. The main concern is that the new help-text sentence is copy-pasted verbatim into five subcommands instead of living once in the parent fs command.

Comment thread cmd/fs/cat.go Resolved
Co-authored-by: Isaac <no-reply@databricks.com>
@vinchenzo-db

Copy link
Copy Markdown
Contributor Author

/review

@review-oss-bot review-oss-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review at 58cdf96. Each finding was verified against the code.

No blocking issues found. 1 non-blocking comment(s) below.

This PR adds a stderr warning when databricks fs commands receive a /Volumes/ or /Skills/ path without the dbfs: scheme, and improves the --help text to clarify that scheme-less paths are always local. The documentation updates are clear and well-placed. The warning's behavior (keeping the path local) is correct, but the suggestion text can mislead macOS users whose /Volumes/ paths are legitimate local mounts.

Comment thread cmd/fs/helpers.go Resolved
Co-authored-by: Isaac <no-reply@databricks.com>
@vinchenzo-db

Copy link
Copy Markdown
Contributor Author

/review

@review-oss-bot review-oss-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review at 9977623. Each finding was verified against the code.

No blocking issues found. 1 non-blocking comment(s) below.

This PR adds a stderr warning when databricks fs commands receive a /Volumes/ or /Skills/ path without the dbfs: scheme, and updates help text to clarify that scheme-less paths are always local. The implementation is clean, well-tested, and the trade-off (warn, don't reject) is reasonable. One side-effect worth addressing: the warning also fires during shell tab-completion, producing noise in interactive sessions.

Comment thread cmd/fs/helpers.go Resolved
Co-authored-by: Isaac <no-reply@databricks.com>
@vinchenzo-db

Copy link
Copy Markdown
Contributor Author

/review

@review-oss-bot review-oss-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review at baadab1. Each finding was verified against the code.

No blocking issues found. 2 non-blocking comment(s) below.

This PR adds a stderr warning when databricks fs commands receive a scheme-less /Volumes/ or /Skills/ path, and improves --help text to clarify that such paths are treated as local. The help-text changes are clean. Two minor issues with the warning: it fires during tab completion (polluting the terminal) and prints twice when fs cp gets two /Volumes/ arguments.

Comment thread cmd/fs/helpers.go Resolved
Comment thread cmd/fs/helpers.go Resolved
Move the scheme-less /Volumes/ and /Skills/ warning out of filerForPath,
which shell completion also calls, into a helper each command runs once
with all of its path arguments.

Co-authored-by: Isaac <no-reply@databricks.com>
@vinchenzo-db

Copy link
Copy Markdown
Contributor Author

/review

@review-oss-bot review-oss-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review at 3bae952. Each finding was verified against the code.

No blocking issues found. 1 non-blocking comment(s) below.

This PR adds a stderr warning when databricks fs commands receive a /Volumes/ or /Skills/ path without the dbfs: scheme, and updates help text to clarify that schemeless paths are always local. The implementation is correct and well-tested (acceptance test + unit test). One minor maintenance concern: the UC prefix list is duplicated within the same file.

Comment thread cmd/fs/helpers.go Resolved
Co-authored-by: Isaac <no-reply@databricks.com>
@vinchenzo-db

Copy link
Copy Markdown
Contributor Author

/review

@review-oss-bot review-oss-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review at 1bb23ae. Each finding was verified against the code.

No blocking issues found. 1 non-blocking comment(s) below.

This PR adds a stderr warning to databricks fs subcommands when a /Volumes/ or /Skills/ path lacks the dbfs: scheme, and updates --help text to clarify that scheme-less paths are local. The change is well-scoped, tested at both the unit and acceptance level, and the doc-comment and code-comment quality is good. One inconsistency with the existing warning mechanism is worth addressing.

Comment thread cmd/fs/helpers.go Resolved

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants