Build AI

MoreSpace

MoreSpace moves photos, video and documents between a phone and the cloud accounts a person already has. Storage is bought per account. One is full, another is nearly empty, and both are being paid for.

Moving that by hand is slow enough that people never do it. At the volume this runs - the infrastructure looks after on the order of 5 petabytes - the problem is knowing what exists where, before a transfer starts.

  • Know what is where
  • Move only what should move
  • Treat the files as private
MoreSpace interface screenshot

Know what is where

An index and a dedupe pass, so the system does not read every byte on every run. The transfer is the last step, not the first.

Indexedknow what exists before moving it

Move only what should move

New content is watched in the background. What already lives elsewhere stays put. The rest goes to the account that has room.

Dedupemove only what is not already there

Treat the files as private

Every movement is authenticated, authorised and encrypted in transit. That is the condition for carrying the files people actually keep.

5 PBprivate files, checked on every move

The build

Indexing before copying

At this scale, reading everything to decide whether to move it does not finish. The analyser shows where the space went - duplicates, files nothing points at - so a person can see it, and the scheduler can act on the index rather than on a guess.

MoreSpace, Indexing before copying

The pipeline has to be quiet

Background sync that surprises someone is a support incident. Filtering what is already stored, then moving the remainder, is how the product stays something people leave running.

MoreSpace, The pipeline has to be quiet

A model inside the pipeline

The model sits in the decisions the pipeline already has to make: what looks like a duplicate, what can wait, what is worth showing. It is not a chat box beside the product. It is how those calls get made, which is the only place a model helps this job.

MoreSpace, A model inside the pipeline

Private files, ordinary rules

There is no later pass where security gets added. Every movement is authenticated, authorised and encrypted, because the product exists to carry private files between accounts. A faster sync that was careless about that would not have been deployable.

MoreSpace, Private files, ordinary rules

How we worked

What moving data asks of us

Scale as an indexing problem

The interesting number is not how fast a file moves. It is knowing the estate well enough to move less of it.

Security on the default path

There is no version of this where the files move first and the policy arrives in a later release.

Intelligence at the decision

A model that describes the product does not move a file. A model in the dedupe and the schedule does.

Technology

What it is built on

  • C#
  • MYSQL
  • Flutter
  • Python
  • Go Lang
  • Elastic Search

Need something like this built?

Tell us what you are trying to ship and an engineer will reply within one business day.