Page History

...

The movers are standalone processes, and communicate with the agent using gRPC, an RPC protocol built using Google's Protocol Buffers.  When the agent starts, it starts the configured movers automatically. 

RPC Interface

service DataMover {
    rpc Register(Endpoint) returns (Handle);
    rpc GetActions(Handle) returns (stream ActionItem);
    rpc StatusStream(stream ActionStatus) returns (Empty);
}

...

The mover register a specific archive id with the agent. The fs_url is the identifier of the filesystem (such as mntent.fsname). This is used to confirm the agent and mover are referring to the same filesystem because all paths passed between the agent and movers are relative paths from the root of the filesystem.

Register returns a handle for a virtual connection with the agent. This handle is used with all the communication with the agent related to with this archive id.

GetActions(Handle) returns (stream ActionItem)

message ActionItem {
    uint64 id = 1; // Unique indentifier for this action, must be used in status messages
    Command op = 2; 
    string primary_path = 3; // Path to primary file (for metadata or reading)
    string write_path = 4; // Path for writing data (for restore)
    uint64 offset = 5; // Start IO at offset
    uint64 length = 6; // Number of bytes to copy
    bytes file_id = 7; // Archive ID of file (provided with Restore command)
    bytes data = 8; // Arbitrary data passed to action. Data Mover specific.
}

GetActions() returns a stream that will provide ActionItems. Each action include a command and parameters to perform on the file referenced in primary_path.  If the action requires writing to the Lustre filesystem (i.e. Restore), then the data must be written to write_path. If a file_id is associated with this file, then it will be included in the ActionItem.

The op can be one of these Commands

ARCHIVE

An Archive command indicates the mover must copy length data starting at offset from the primary_path to the backend. An identifier used to refer to data stored in the backend can be returned as a file_id in the final ActionStatus message.

RESTORE

A Restore command indicates the mover must copy length data starting at offset from the backend to the write_path. The file_id provided by the mover after the Archive is included.

REMOVE

A Remove command indicates the file in primary_path is no longer managed HSM and the copied data should be removed from the backend.

StatusStream(stream ActionStatus) returns (Empty)

message ActionStatus {
    uint64 id = 1; // Unique identifier for action
    bool  completed = 2; // True if this is last update
    int32  error = 3; // Non-zero indicates an error
    uint64 offset = 4; 
    uint64  length = 5;
    Handle handle = 6;
    bytes file_id = 7; // Included with completion of Archive
    int32 flags = 8; // Additial flags (used for errors only?)
}

StatusStream() creates a stream that is used to send ActionStatus messages. The messages are used to send completion notificationand (for long running actions) progress updates.

When completed is 0, the message is a progress update for the message id, and the length value will include the amount of data that has been copied since the previous update. This is useful for long running transactions to provide update for monitoring systems, and also prevents the action from being timed out and restarted.  It is important for movers to send these messages periodically even when no data is being copied (such waiting for a tape to be loaded) to ensure the action does not time out.

When completed is non-0, the message indicates that message referenced by ID has completed. If error is non-zero, then the action has failed with given error code. If error is 0, then the action has succeed, and the offset and length range of data has been copied by the mover. If file_id is included, then the agent will store this and include this with future actions referring to this file. 

POSIX Mover

S3 Mover