Skip to content

Articles

The Zero-Knowledge Illusion in Cloud Transfer Tools

The zero-knowledge illusion in cloud transfer tools, encrypted in transit versus never seen

A transfer tool can only call itself zero-knowledge if it is never in a position to see your files or your credentials. There is really only one way to guarantee that: the tool never sits in the path your data travels. If it runs on someone else's servers, "zero-knowledge" describes an intention, not the architecture.

"Encrypted in Transit" Is Not the Whole Story

Section titled ""Encrypted in Transit" Is Not the Whole Story"

Most hosted tools say your data is encrypted in transit, and it is. The detail that matters is where the encryption stops. When data moves through a company's servers, it is decrypted there so it can be read and sent on to the destination, then encrypted again for the second leg. That is hop-by-hop encryption, not end-to-end. For a moment, on a machine you do not control, your files are in the clear.

This is not a sign of bad intent. It is how a relay has to work. But it means "encrypted in transit" and "we never see your files" are two different promises, and only one of them is being made.

The Bigger Exposure Is Your Credentials

Section titled "The Bigger Exposure Is Your Credentials"

Files in transit are the visible worry. The quieter one is the keys. To move data on your behalf, a tool needs your storage credentials, and a storage key is not a limited transfer pass. It can read, write, list, and delete across your account.

A hosted service has to store those keys somewhere so it can use them. That store, holding the credentials of many users, is a single valuable target. If it is breached, the exposure is not one transfer; it is standing access to everyone's storage.

One arrangement actually delivers it: the tool runs on your own machine, keeps your credentials in local storage, and connects straight to your providers. With nothing in the middle, there is no relay to decrypt your files and no shared vault to breach. The vendor's service knows nothing because it is not in the transfer path.

Blober works this way. Credentials stay in a local store on your computer, the data streams directly between your providers and your machine, and there is no Blober account or server in the path.

You do not have to take a claim on faith. A few questions sort the architecture out:

  • Does it require an account with the tool's own service? A pure local tool needs a licence, not an account that holds your data.
  • Where are credentials stored? On your machine, or on the tool's servers?
  • Can it run when the tool vendor's own servers are unreachable? If the core transfer can run with those servers unreachable, they are not in your data path. If it cannot, they are.

That vendor-offline test is the most telling. A tool that keeps working with its own service unreachable cannot be sending your files through that service.

The command-line tool rclone is also local, and that is its strength: it runs on your machine and moves data directly between providers. Its credentials live in a configuration file on your disk, which rclone says should be protected because it typically contains login information[1]. Blober keeps the same local-only principle while removing the configuration step, storing your credentials on your own machine and connecting straight to your providers.

Zero-knowledge is about the middleman, not the endpoints. No tool can protect you from a provider you have chosen to authorize; that provider can see what is in its own account by definition. What a local-first tool removes is the extra party, the one that had no need to see your data and no business holding your keys.

Is "encrypted in transit" enough? It protects data from outsiders on the wire, but not from the service doing the relay, which decrypts it to pass it along. End-to-end privacy requires that no middle service ever holds the unencrypted data.

Why are credentials a bigger deal than the files? A single transfer exposes one set of files. A leaked storage key exposes the whole account, for as long as the key stays valid.

How do I know a desktop tool is not phoning home? Test it offline. If the transfer between your clouds runs with the tool's own servers unreachable, your data is not passing through them.

What 'Local-First' Actually Means

What local-first software means, software that runs on your device rather than a remote server

Local-first software runs on your own device and keeps your data and its core features working without depending on someone else's servers. The cloud is still welcome, but it is optional rather than required. You hold the data, the app and local state remain usable offline, and nothing essential disappears if a company does.

The phrase was popularized by a 2019 essay from the research group Ink & Switch, titled "Local-first software: you own your data, in spite of the cloud"[1]. It set out a handful of ideals for software that respects the person using it: your data stays on your device and remains yours, the work is available offline, it lasts for the long run instead of vanishing when a service shuts down, and privacy is the default rather than an upgrade.

The essay was written about documents and collaboration, but the principles travel well to any tool that touches your data, including one that moves files between clouds.

It helps to place local-first between two older ideas.

  • Cloud-first software lives on a company's servers. You reach it through a browser, your data sits in its database, and when the service is down or gone, so is your access.
  • Local-only software is the classic desktop app that could not talk to anything else. Your data was yours, but it was stranded on one machine.

Local-first keeps the good parts of both. Your data and the app live on your device, so you keep control and offline access, and the app still reaches the cloud when you want it to. The difference from cloud-first is who is in charge: the cloud serves you, instead of holding you.

Applied to moving files between clouds, local-first has a clear shape:

  • The app runs on your computer, not in a browser tab on someone else's servers.
  • It connects directly to your providers, so your files are not relayed through a middle service.
  • Your credentials stay in a local store on your machine.
  • The core work does not depend on the tool's own servers, so it keeps running even when they are unreachable.
  • It keeps working for the long run. Blober is a lifetime licence with future updates under the current terms, so the copy on your machine does not stop working when a billing cycle ends.

This is the same idea described in Your Files, Your Machine, No Middleman, set out as a principle here rather than step by step.

  • Ownership. Your files and your keys stay on your side. A tool you run cannot quietly change what it does with data it never receives.
  • Longevity. A local-first tool does not depend on a company staying in business to keep functioning. What works today keeps working.
  • Privacy. With no middle service in the path, there is no extra party to see your files or hold your credentials.
  • No lock-in. Because the tool moves data between the storage you already use, it makes leaving any one provider easier, not harder.

Local-first is not anti-cloud, and it is not a claim that servers are bad. Plenty of good software is cloud-first for good reasons. Local-first is a statement about control: the data and the core features belong on your device, and the cloud is something you reach out to on your terms. For a tool whose whole job is handling your files, that is a sensible default.

Is local-first the same as offline? Offline is one of its results, not the whole idea. Local-first means the app and your data live on your device; working offline follows from that.

Does local-first mean I cannot use the cloud? No. It means the cloud is optional for the app to function. You still connect to cloud providers; you are just not dependent on the tool's own servers.

How is this different from an old desktop program? An old desktop program was often local-only, stranded on one machine. Local-first keeps your data on your device and still connects to the cloud when you want it.

Where Cloud Transfer Tools Actually Send Your Data

Where cloud transfer tools send your data, hosted service versus local script versus desktop app

When you move files between clouds, the tool you pick decides the route your data takes. There are three common designs: a hosted service that relays your files through its own servers, a local command-line tool that runs on your machine, and a local desktop app. They differ most in two places you cannot see from the marketing page: where your files travel, and where your credentials live.

You sign in on a website, connect your clouds, and the service moves the data on its servers. It is convenient and needs nothing installed. In exchange, your files pass through infrastructure you do not control, your credentials are stored on its systems, and the work depends on its uptime, its pricing, and its jurisdiction. Pricing here is usually a subscription or a per-GB transfer fee, sometimes with a monthly cap.

Tools like rclone run on your own machine and move data directly between providers. Your files do not detour through anyone else's servers, and your credentials sit in a local file you control; rclone documents cloud authentication in its local configuration file and recommends protecting that file because it contains login information[1]. The cost is your time: configuration files, keys to manage, and provider-specific flags, all from a terminal.

A desktop app keeps the direct, on-your-machine route of a command-line tool and adds a visual interface. You connect a provider by signing in, browse your files, pick a destination, and run. Credentials stay in a local store, the data streams straight between your providers and your machine, and there is no Blober account or Blober server in the transfer path. Blober is built this way, with a lifetime licence rather than a subscription.

QuestionHosted serviceLocal scriptLocal desktop app
Where do your files travel?Through the company's serversDirectly, through your machineDirectly, through your machine
Where do your credentials live?On the company's systemsIn a local file you controlIn a local store on your machine
Does the tool vendor need to be in the transfer path?YesNoNo
Can provider limits still apply?Yes, and shared company servers can be throttled or blocked for every user at onceYes, but traffic uses your own IPYes, but traffic uses your own IP
What does it cost you over time?Subscription or per-GB feeFree tool, your setup timeLifetime licence, no subscription or per-transfer fees
What does it ask of you?An account and trust in the relayComfort with a terminalSigning in and clicking

A hosted service reaches every provider from the same pool of server addresses. That is the pattern most likely to attract shared-service rate limits, geofencing, or blocks, and when it happens, every user of that service is affected at once. A tool on your own machine connects from your own IP address, so there is no company relay address to single out. For services that were never designed for bulk export, that difference can matter, although account-level and API limits still apply.

  • Pick a hosted service if you want nothing installed, you are moving a modest amount, and a shared team dashboard matters more than keeping data and keys on your own side. These services are genuinely convenient, and for the right job that convenience wins.
  • Pick a local script if you live in the terminal and want maximum control with no interface in the way.
  • Pick a desktop app if you want the direct, local route without the setup, especially for large libraries or providers that resist bulk transfers.

No single design is best for everyone. A hosted service removes all setup and is easy to share across a team. A script is the most flexible if you are willing to learn it. A desktop app aims at the middle: the privacy and reach of running locally, with the ease of a visual tool. The right choice is the one whose trade-offs you are happy to live with.

Does a desktop app still send my data through my computer? Yes, and that is the point. The data streams through your machine to the destination instead of through a separate company's servers, and no second full copy is saved to your disk.

Why can a hosted service get blocked when a desktop app does not? The hosted service connects from shared company servers that a provider can recognize and throttle. A desktop app connects from your own IP, so there is no shared relay address to block for every user at once.

Is a local tool less safe because the credentials are on my machine? Your machine is a single device under your control, not a shared database of many users' keys. For most people, one device they control is a smaller target than a company server holding thousands of accounts' credentials.

Archiving Large Research and Scientific Datasets Across Clouds

Archiving large research and scientific datasets across clouds

Research datasets are large, occasionally needed years later, and often spread across storage paid for by different grants or collaborators. Archiving them well means picking durable storage, moving the data without a scripting project, and using transfers that resume when a multi-day run gets interrupted.

Anyone who has managed a lab's data knows the pattern. A dataset lives on a cluster's object store, a copy sits in a collaborator's account, and the grant that funded the original storage is ending. The data has to move, it is enormous, and nobody wants to own the migration.

Choose Storage That Suits an Archive

Section titled "Choose Storage That Suits an Archive"

Active analysis and long-term archive have different needs. For the archive, the priorities are durability and a cost model that fits data you read back rarely:

  • Object storage such as Backblaze B2, Wasabi, or Cloudflare R2 is built for exactly this: large objects, high durability, S3-compatible so your existing tools work.
  • Compare on egress and minimum storage duration, not the headline rate. For an archive you touch a few times a year, those terms decide the real cost far more than the storage price.
  • Keep a second copy. A single archive is one copy. Durable does not mean infallible, and a second location is what the 3-2-1 rule is for.

Moving the Data Without a Scripting Project

Section titled "Moving the Data Without a Scripting Project"

The usual options at this scale are command-line tools and custom scripts, which is fine if you have an engineer to spare and a problem when you do not. The bottleneck is rarely the copy itself. It is listing millions of small files, keeping throughput up with parallelism, and resuming cleanly when a run that takes days gets interrupted.

Blober handles those parts from a desktop app. It connects to S3, B2, Wasabi, R2, DigitalOcean Spaces, Azure Blob, and local storage, copies between them directly without staging a full copy on disk, runs transfers in parallel, and has skip-existing so a paused or failed run picks up where it left off instead of starting over. For a dataset larger than any one machine's disk, that combination is the difference between a finished archive and an abandoned one.

An archive nobody can navigate is only half useful. As you move data, keep a simple record: what went where, when, and the rough file count, so a future you or a future student can find a dataset without reverse-engineering the folder tree. A short README in the destination bucket pays for itself the first time someone needs the data after you have moved on.

Where should I archive large research datasets? Durable object storage such as Backblaze B2, Wasabi, or Cloudflare R2, chosen on egress and minimum storage duration rather than the headline rate, with a second copy in another location.

How do I move a multi-terabyte dataset between clouds? Use a tool that transfers directly, runs in parallel, and resumes. Blober copies between object stores and local storage from a desktop app, with skip-existing so interrupted runs continue rather than restart.

What makes large transfers fail? At scale, listing millions of small files and surviving interruptions are the hard parts, not the copy. Parallelism and resumable, skip-existing transfers are what get a multi-day run to finish.

Is object storage good for research archives? Yes. It is durable, built for large objects, and usually S3-compatible, so existing tools work. Keep a second copy elsewhere to satisfy 3-2-1.

Move multi-terabyte datasets between object stores and local storage without a scripting project. Blober transfers in parallel, preserves structure, and resumes interrupted runs.

Download Blober at blober.io

Cloudflare R2 for AI Training Data: Why Zero Egress Changes the Math

Cloudflare R2 as a home for AI training data, with zero egress on repeated reads

Why Egress Is the Hidden Tax on Training Data

Section titled "Why Egress Is the Hidden Tax on Training Data"

Training a model means reading the same dataset over and over, once per epoch, often from GPUs that sit outside your storage provider's network. On most object stores you pay an egress fee every time that data leaves the bucket. Cloudflare R2 does not charge egress fees, so reading a dataset a hundred times costs the same in transfer as reading it once. For read-heavy AI work, that quietly changes the math.

People size storage by the price per terabyte and then get surprised by the transfer line on the bill. For an archive you rarely open, egress barely matters. For a training set you stream through a data loader thousands of times, egress is the cost.

What Makes Training Data Different From an Archive

Section titled "What Makes Training Data Different From an Archive"

Training data has a few traits that make egress the deciding factor:

  • It is read many times. Every epoch reads the whole set again. Hyperparameter sweeps and multiple runs multiply that.
  • It is large. Image, video, audio, and text corpora run to terabytes, and embeddings pile on more.
  • The compute is often elsewhere. GPUs in another cloud or a rented cluster mean the data crosses a network boundary on every read, which is exactly what egress charges for.

Put those together and a metered-egress store can cost more to read than to hold.

Two properties do the work. First, R2 does not charge egress fees, so repeated reads from outside Cloudflare do not accumulate transfer cost. Second, R2 is S3-compatible, so the data loaders, SDKs, and tools your pipeline already uses point at it by changing the endpoint and the keys. You do not rewrite your training code to adopt it.

A couple of honest caveats, because the math is not free in every direction. R2 has its own operation and request considerations, and throughput depends on how your loader and network are set up. If your training compute lives in the same cloud as your current data, reads inside that cloud may already avoid egress, so R2's advantage is largest when storage and compute would otherwise sit on different networks. Confirm Cloudflare's current terms before you commit a pipeline to them.

A training corpus rarely starts life in one place. It is scraped to a local disk, staged in an S3 bucket, or scattered across a few accounts from different collaborators. Consolidating it into one R2 bucket is the setup step.

Blober moves data into R2 directly from AWS S3, Backblaze B2, Wasabi, DigitalOcean Spaces, Azure Blob, Dropbox, Google Drive, or local storage. It copies in parallel, keeps the folder structure intact, and has skip-existing, so the first run stages the whole corpus and later runs only carry the new files as the dataset grows. You are not downloading the set to a laptop and pushing it back up, which matters when the corpus is bigger than any one machine's disk.

  1. Choose R2 as the dataset home if your training compute reads it repeatedly from outside Cloudflare.
  2. Stage the corpus into an R2 bucket with Blober, in parallel and with structure preserved.
  3. Point your S3-compatible data loader at the R2 endpoint and train.
  4. Re-run Blober with skip-existing as you add data, so only the new files move.

Keep a second copy somewhere else as well. One bucket is one copy, and the 3-2-1 rule applies to a dataset you cannot easily recreate just as much as to family photos.

Does Cloudflare R2 charge egress fees? No. R2 does not charge egress fees for reading your data out, which is its main draw for read-heavy workloads like model training. Confirm the current terms on Cloudflare's site before committing.

Is Cloudflare R2 good for machine learning datasets? Yes, especially when your training compute reads the dataset repeatedly from outside Cloudflare's network. Zero egress removes the per-read transfer cost that dominates training storage bills.

Is R2 S3-compatible for data loaders? Yes. R2 exposes an S3-compatible API, so existing S3 data loaders, SDKs, and tools work by changing the endpoint and credentials.

How do I move my training data into R2? Use a tool that transfers directly and in parallel. Blober stages datasets into R2 from S3, B2, Wasabi, Spaces, Azure Blob, and local storage, with skip-existing for incremental updates.

Stage your training data into R2 without a scripting project. Blober moves datasets into R2 from S3, B2, Wasabi, Spaces, Azure Blob, and local storage, in parallel and with structure intact.

Download Blober at blober.io