ScoutFS is an open source Linux clustered filesystem developed specifically to address the unique and challenging requirements of mass storage and archiving workloads. Better yet, it introduces many new capabilities by radically increasing the number of POSIX files and objects that can reliably be maintained and searched within a global namespace.
ScoutFS supports mass storage and archiving applications by managing hundreds of billions of metadata records. There is no central metadata controller in a ScoutFS cluster, nor any single point of failure. Metadata is processed on all nodes or a subset of nodes in a ScoutAM cluster.
The ScoutFS global namespace is designed to accommodate hundreds of billions of files and objects while delivering responsive search and query results. Mass storage capacity is greater than 100 exabytes per system.
ScoutFS stores metadata on a separate device from data. Metadata separation allows ScoutFS to direct extremely high metadata IOPS workloads to smaller higher performance NVMe or SSD LUN’s, while allocating data reads and writes to different devices that are optimized for streaming throughput. In extreme-scale environments, the metadata storage may be directed to a dedicated controller to ensure maximum performance.
ScoutFS is an open source GPLv2, in-kernel Linux filesystem. Open source metadata combined with open source data formats gives customers complete control over data collections and aligns with long-term data preservation and autonomy goals.
ScoutFS scales horizontally by adding server nodes. Each node in a ScoutFS cluster participates in metadata and data management, enabling the system to achieve higher levels of performance with additional compute resources.
The ScoutFS scale out architecture allows nodes to enter or leave the cluster without disruption. Due to the journaled and atomic design, work is never lost during a node failure. Failover within a cluster is fully managed without the need for external HA frameworks.
Atomic metadata and data sequence numbers are indexed so both inode attribute changes and file content changes may be quickly discovered within the system. Indexed change sequence information eliminates the need for routine file system scans.
ScoutFS supports user defined and API driven custom metadata tagging, which enables the creation and management of enhanced metadata. Enhanced metadata is highly scalable and can be searched with responsive performance even among very large file and object populations.
ScoutFS supports fast metadata dumps for greater convenience and enhanced system protection. Metadata restores benefit from extremely high file creation rates in the ScoutFS multi-node system. Incremental file system dumps avoid wasted processing time by skipping items that have not changed.
The ScoutFS cluster architecture takes advantage of technologies that have emerged in recent years including the commercialization of SSD and NVMe devices.
ScoutFS stores metadata and data on separate devices. This separation allows low capacity but high performance metadata workloads to be concentrated on relatively small quantities of NVMe or SSD devices, while data is stored on either SSDs or conventional disk arrays optimized for streaming read/write performance.
Connect with Versity today to find out how we can tailor a solution to keep your organization’s data safe and accessible as you advance your mission.