`copyurl` should respect `--size-only` as much as possible
Before you start
- I have searched the forum and the existing issues for this problem.
- I have tested with the latest beta or stable release and can still reproduce the problem.
- This is a reproducible bug, not a usage question (questions belong on the forum).
Associated forum post URL
No response
What is the problem you are having with rclone?
I humbly believe that copyurl should respect --size-only as much as possible.
For example, when the Content-Length is available and --size-only is given, copyurl should use Content-Length, compare it with that of the file-name on the remote, and skip the download altogether if the sizes are equal.
Currently, given a tuple <URL,FILENAME>, copyurl will always download that URL to the remote FILENAME, even if (--size-only was given) && (the FILENAME is already present on the remote) && (the Content-Length of the URL is equal to the size of the FILENAME)
rclone version
rclone v1.75.0
- os/version: linuxmint 22.1 (64 bit)
- os/kernel: 6.8.0-51-generic (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.26.5
- go/linking: static
- go/tags: none
Operating system
Linux
Which cloud storage system are you using?
SFTP
Output of rclone config redacted
[sftp]
type = sftp
host = XXX
user = XXX
port = 40000
pass = XXX
use_insecure_cipher = true
The command you were trying to run
`rclone copyurl --size-only --urls upload.csv sftp:/user/local`
with `upload.csv` containing one line pertaining to a PDF *that was already on the remote's `/user/local`*:
`https://<some-url-erased-for-security>/template.pdf,template.pdf`
A log from the command with the -vv flag
2026/09/09 12:40:46 DEBUG : rclone: Version "v1.75.0" starting with parameters ["rclone" "copyurl" "--size-only" "--urls" "upload.csv" "sftp:/user/local" "-vv"]
2026/09/09 12:40:46 DEBUG : Creating backend with remote "sftp:/user/local"
2026/09/09 12:40:46 DEBUG : Using config file from "<redacted>"
2026/09/09 12:40:46 NOTICE: sftp: No host key validation is being performed. Set known_hosts_file (to "none" to silence this notice) or use --sftp-pin-host-key to enable it. See: https://rclone.org/sftp/#host-key-validation
2026/09/09 12:40:48 DEBUG : sftp://<redacted>:40000//user/local: New connection <redacted> to <redacted>
2026/09/09 12:40:50 DEBUG : sftp://<redacted>:40000//user/local: Running shell type detection remote command: echo ${ShellId}%ComSpec%
2026/09/09 12:40:50 DEBUG : Saving config "shell_type" in section "sftp" of the config file
2026/09/09 12:40:54 DEBUG : sftp://<redacted>:40000//user/local: Using root directory "/user/local"
2026/09/09 12:40:55 DEBUG : sftp://<redacted>:40000//user/local: Checking default md5 hash commands
2026/09/09 12:40:56 DEBUG : sftp://<redacted>:40000//user/local: Running remote command: md5sum
2026/09/09 12:40:56 DEBUG : sftp://<redacted>:40000//user/local: Hash command skipped: failed to run "md5sum": : Process exited with status 1
2026/09/09 12:40:56 DEBUG : sftp://<redacted>:40000//user/local: Running remote command: md5 -r
2026/09/09 12:40:57 DEBUG : sftp://<redacted>:40000//user/local: Hash command skipped: failed to run "md5 -r": : Process exited with status 1
2026/09/09 12:40:57 DEBUG : sftp://<redacted>:40000//user/local: Running remote command: rclone md5sum
2026/09/09 12:40:59 DEBUG : sftp://<redacted>:40000//user/local: Hash command skipped: failed to run "rclone md5sum": : Process exited with status 1
2026/09/09 12:40:59 DEBUG : sftp://<redacted>:40000//user/local: Checking default sha1 hash commands
2026/09/09 12:40:59 DEBUG : sftp://<redacted>:40000//user/local: Running remote command: sha1sum
2026/09/09 12:41:00 DEBUG : sftp://<redacted>:40000//user/local: Hash command skipped: failed to run "sha1sum": : Process exited with status 1
2026/09/09 12:41:00 DEBUG : sftp://<redacted>:40000//user/local: Running remote command: sha1 -r
2026/09/09 12:41:00 DEBUG : sftp://<redacted>:40000//user/local: Hash command skipped: failed to run "sha1 -r": : Process exited with status 1
2026/09/09 12:41:00 DEBUG : sftp://<redacted>:40000//user/local: Running remote command: rclone sha1sum
2026/09/09 12:41:01 DEBUG : sftp://<redacted>:40000//user/local: Hash command skipped: failed to run "rclone sha1sum": : Process exited with status 1
2026/09/09 12:41:01 DEBUG : sftp://<redacted>:40000//user/local: Setting hash command for md5 to "none" (set md5sum_command to override)
2026/09/09 12:41:01 DEBUG : Saving config "md5sum_command" in section "sftp" of the config file
2026/09/09 12:41:05 DEBUG : sftp://<redacted>:40000//user/local: Setting hash command for sha1 to "none" (set sha1sum_command to override)
2026/09/09 12:41:05 DEBUG : Saving config "sha1sum_command" in section "sftp" of the config file
2026/09/09 12:41:46 INFO :
Transferred: 9.594 MiB / 29.668 MiB, 32%, 239.742 KiB/s, ETA 1m25s
Transferred: 0 / 1, 0%
Elapsed time: 50.8s
Transferring:
* bug-copyurl/template.pdf: 32% / 29.668 MiB, 239.742 KiB/s, 1m25s
2026/09/09 12:42:11 DEBUG : sftp://<redacted>:40000//user/local: Not closing 0 unused connections as 1 sessions active
2026/09/09 12:42:46 INFO :
Transferred: 26.094 MiB / 29.668 MiB, 88%, 292.365 KiB/s, ETA 12s
Transferred: 0 / 1, 0%
Elapsed time: 1m50.8s
Transferring:
* bug-copyurl/template.pdf: 88% / 29.668 MiB, 292.363 KiB/s, 12s
2026/09/09 12:43:00 INFO :
Transferred: 29.668 MiB / 29.668 MiB, 100%, 196.894 KiB/s, ETA 0s
Transferred: 1 / 1, 100%
Elapsed time: 2m9.2s
Anything else?
Also, and quite unlike copy and copyto, the command copyurl won't actually attach a .partial* suffix to a file it is uploading to the remote. I humbly believe this one could actually be reported in a standalone issue!
Source: rclone/rclone