Takeout

[Tristan-C789]

CERNBox Takeout

Users currently have no convenient, self-service way to obtain a full, offline copy of everything they store in their CERNBox personal space. This is needed for personal backups, offboarding (e.g. leaving CERN), and data-access requests. "Takeout" lets a user trigger an export from the web interface; the system packages their space into one or more archives asynchronously, stores them in a pre-configured EOS location, and notifies the user by email with download links when the export is ready. Exports expire automatically after a configurable retention period.

Features

This PR mplements the core takeout logic as a Reva background job. Given a user, a max archive size, and a format, the job collects the contents of the user's personal space, packages them into one or more archives of the requested format, splitting output so that no single archive exceeds the configured max size, and uploads the artifact(s) to a pre-configured EOS folder.

It exposes two HTTP endpoints: one to create a new takeout job (taking max size and format), and one to get the status of the current/last takeout for the authenticated user. The create endpoint enforces one active job per user, if a takeout is already running, it returns the existing job rather than starting a new one. The status endpoint returns the job state and, when completed, the download link(s) and expiry. In particular:

• POST endpoint creates a takeout for the authenticated user with body {max_archive_size, format}; unsupported size/format is rejected with a clear error.
• If the user already has an active takeout, the create endpoint returns the existing job's id/status instead of starting a second one.
• GET status endpoint returns the current state (none / queued / running / completed / failed) and progress if available.
• When completed, the status response includes the download link(s) and expiry time.
• When failed, the status response includes an error/cause indication.

[update-docs]

Thanks for opening this pull request! The maintainers of this repository would appreciate it if you would create a changelog item based on your changes.

[diocas]

I didn't check the archiver code, but I have the impression that this is a copy of it. Didn't we discuss that we would integrate the current archive code in the takeout, so that we only set the logic in a single place? The advantage of the takeout would be the possibility to keep track of error instead of just killing the connection. And the fact that we can chunk, but maybe we can achieve a single logic for both (infinite chunk size would have the same output as the archiver, no?)

[diocas]

Shall we ignore/skip errors with files and write them (the error) to a file (that the user can check)?

[jessegeens]

Should never be the case, but make sure ContainerRes.Status != nil, otherwise you get a panic.