Site Logo

High-Performance Computing

ScoutAM is Versity’s advanced data management solution designed to optimize data storage and access in High-Performance Computing (HPC) environments. By leveraging cutting-edge technology, ScoutAM enhances the efficiency, scalability, and reliability of data handling, addressing the unique challenges faced by HPC sites.

Request a Briefing

Accelerating HPC Workflows with Advanced
Mass Storage Solutions

In today’s high-performance computing landscape, organizations are grappling with datasets that grow exponentially. To remain competitive, these sites must efficiently and cost-effectively manage this ever-expanding data. The unique challenges of scale, complexity, and technological evolution in high-performance computing demand robust solutions. Our software product excels by tackling these critical issues head-on, offering advanced solutions that enhance performance and drive down costs, ensuring your data management is both efficient and future-proof.

Massive and
Diverse Datasets

HPC systems generate and
handle vast datasets from varying sources, including simulations and global research initiatives. ScoutAM efficiently and seamlessly manages, integrates, and retrieves this diverse data ensuring seamless access

Complex
Workflows

As HPC environments push towards exascale computing, workflows are complex and performance is more critical than ever. ScoutAM is designed to meet these scale and performance demands, providing the speed, scalability, and efficiency required for exascale HPC.

Balance Speed
and Cost

ScoutAM empowers organizations to effectively leverage different storage tiers, from ultra-fast NVMe/flash storage to cost-efficient tape for long-term archival and everything in between. ScoutAM ensures the right data is in the right place at the right time.

Rapidly Evolving
Technology

The field of HPC is continuously evolving, with new technologies and methodologies emerging regularly. ScoutAM enables organizations to keep up with these advancements and integrate them into existing systems without disrupting ongoing operations.

Key Benefits

Cost Savings

HPC sites can significantly reduce costs by seamlessly offloading data from high-performance storage to more affordable warm or cold storage, retrieving it automatically when needed. This process is completely transparent to users, providing seamless access without the high costs associated with high capacity primary storage. By utilizing less expensive storage tiers without sacrificing accessibility, HPC sites can better manage their budgets while maintaining peak performance. Plus, with subscription pricing not tied to node counts, core counts, or data transfer amounts, scaling is worry-free.

High Performance

Architected for exascale workloads, ScoutAM is capable of ingest and mass storage management in parallel at high speed. The performance is orders of magnitude greater than that of our competitors, making ScoutAM a leading choice for the most demanding HPC environments.

Enhanced Metadata Performance

ScoutAM significantly reduces metadata lookup times, ensuring rapid data access and retrieval, which is critical in HPC environments where large datasets are common. ScoutAM’s advanced indexing and search capabilities streamline operations, allowing sysadmins to focus more on their work and less on data management.

Ultimate Disaster Recovery

With robust error handling and recovery mechanisms, ScoutAM ensures data integrity and availability even in the face of hardware failures or other issues. Its high-availability architecture provides continuous operation and minimal downtime, crucial for mission-critical HPC applications.

Comprehensive and Personalized Support

Users do not need any special tools or training to use ScoutAM. Access is automatic and seamlessly integrated into their existing workflows, utilizing standard tools they use every day. The system supports all users within the organization, as well as external users if applicable. It is compatible with all major access protocols and tools, including Globus, S3, NFS, Samba, scp/rsync, and more, ensuring versatile and secure data access for diverse user needs.

EXA-Scale with ScoutAM

Designed to scale with your data, ScoutAM scales both horizontally and vertically as needed. It is deployed on a cluster of nodes and metadata is processed independently so it can handle exabytes of information seamlessly. This scalability ensures that as your HPC site grows, ScoutAM grows with it. The flexibility of ScoutAM allows for integration with various storage systems and workflows, making it a versatile solution for diverse HPC needs.

Learn more

Flexible and Efficient Data Movement

HPC sites can choose between two distinct configurations—cross-mounting with a parallel filesystem or utilizing the S3 interface—or combine them for added flexibility and scalability in their storage infrastructure.

Seamless File Data Management:

  • HPC sites often cross-mount ScoutAM and the Parallel Filesystem on I/O nodes or data movers. Users see the archive as just another directory, allowing them to utilize familiar tools like rsync, scp, and more to manage their data.
  • ScoutAM seamlessly integrates with Globus, enabling efficient and secure data transfers across high-performance computing environments.
  • Prologue and epilogue scripts can be used with job schedulers to ensure data is in the right place at the right time, further enhancing workflow efficiency.

Utilizing the Filesystem S3 Interface

Alternatively, HPC sites can leverage the S3 interface to connect VAST or WEKA’s high-performance parallel file systems to Versity’s ScoutAM. This configuration enables a high performance object to tape workflow. VAST and WEKA’s high-performance S3 output enables data movement from the filesystem to ScoutAM, optimizing and automating the process of moving data.

Hear From Peers How Versity’s ScoutAM Makes a Difference

Play

Seamless S3 Data Ingest for Massive Archive

Previously, PAWSEY Supercomputing Centre, one of Australia’s leading HPC facilities, struggled with a solution that couldn’t handle S3 data ingestion, limiting their ability to effectively use object storage in front of their massive archive. Faced with an ever-growing data repository, PAWSEY chose Versity’s ScoutAM to manage both existing and new data, seamlessly integrating object storage into their workflows. Thanks to ScoutAM’s zero data migration feature, PAWSEY was able to quickly convert their existing archive without the need for a time-consuming and costly data migration. ScoutAM’s automatic and transparent data management features eliminated the need for extensive training, fitting effortlessly into PAWSEY’s operations. This strategic move not only reduced administrative overhead and operational costs but also empowered PAWSEY to focus more on driving scientific research and innovation.

Learn more
Play

Scalable Data Intake Across Diverse Sources

The National Center for Supercomputing Applications (NCSA) encountered significant challenges in managing the enormous volumes of data generated by their diverse scientific projects. As the scale and complexity of their data grew, so did the need for a robust and adaptable storage solution. ScoutAM’s high-speed access and scalable architecture was pivotal in enhancing NCSA’s data storage capabilities. Its advanced features addressed the growing demands by providing seamless performance and reliability. ScoutAM’s support for multiple access protocols, including Globus, S3, NFS, and more, ensures versatile and secure data access for all users, both internal and external. This integration optimized performance and facilitated more efficient collaborative research efforts. By streamlining data management and improving accessibility, ScoutAM helped NCSA advance their scientific objectives with greater efficiency and effectiveness.

Learn more

Zero Data Migration

ScoutAM can read existing data formats, eliminating the need for extensive data migrations. This capability reduces downtime and complexity, allowing HPC sites to integrate ScoutAM into their existing infrastructure seamlessly. By supporting a wide range of data formats, ScoutAM ensures that archival data is managed efficiently without disrupting existing workflows. The non-destructive conversion process preserves the integrity of your data while enabling advanced features and seamless integration into your current systems. Here’s how the conversion process works:

Conversion Process

1. Prepare the Existing System

Quiesce the system to suspend user access and prevent new data from being added.

2. Export Metadata

Export metadata using system-specific commands, capturing file names, paths, permissions, and media locations.

3. Import Metadata to ScoutAM

Use ScoutAM’s built-in commands to import the collected metadata, preserving file structures and permissions.

4. Catalog Media

Add the archival media to ScoutAM’s catalog. The media will be recognized as read-only, ensuring data integrity and backward compatibility.

5. Redirect User Access

Redirect all user access to ScoutAM. Data can be accessed and written using modern S3 or traditional POSIX interfaces. All data that is written by ScoutAM is in an open tar format

Rise to the challenge

Connect with Versity today to find out how we can tailor a solution to keep your organization’s data safe and accessible as you advance your mission.

Request a Briefing