项目作者: Azure

项目描述 :
Simplify HPC and Batch workloads on Azure
高级语言: Python
项目地址: git://github.com/Azure/batch-shipyard.git
创建时间: 2016-08-26T01:33:23Z
项目社区:https://github.com/Azure/batch-shipyard

开源协议:MIT License

下载


Build Status
Build Status
Build status

Batch Shipyard

PROJECT STATUS

This toolkit is no longer actively maintained. The develop branch has
proposed fixes for outstanding issues, but they will not be merged back to
master. Please see the main Azure Batch
GitHub repository for more information about Azure Batch.

dashboard

Batch Shipyard is a tool to help
provision, execute, and monitor container-based batch processing and HPC
workloads on
Azure Batch. Batch Shipyard
supports both Docker and
Singularity containers. No experience with the
Azure Batch SDK is needed; run
your containers with easy-to-understand configuration files. All Azure
regions are supported, including non-public Azure regions.

Additionally, Batch Shipyard provides the ability to provision and manage
entire standalone remote file systems (storage clusters)
in Azure, independent of any integrated Azure Batch functionality.

Major Features

Container Runtime and Image Management

Data Management and Shared File Systems

Monitoring

Open Source Scheduler Integration

Azure Ecosystem Integration

Azure Batch Integration and Enhancements

  • Federation
    support: enables unified, constraint-based scheduling to collections of
    heterogeneous pools, including across multiple Batch accounts and Azure
    regions
  • Support for simple, scenario-based pool autoscale
    and autopool to dynamically scale and control computing resources on-demand
  • Support for Task Factories
    with the ability to generate tasks based on parametric (parameter) sweeps,
    randomized input, file enumeration, replication, and custom Python code-based
    generators
  • Support for
    multi-instance tasks
    to accommodate MPI and multi-node cluster applications packaged as Docker or
    Singularity containers on compute pools with automatic job completion and
    task termination
  • Seamless, direct high-level configuration support for popular MPI runtimes
    including OpenMPI, MPICH, MVAPICH, and Intel MPI with automatic configuration
    for Infiniband, including SR-IOV RDMA VM sizes
  • Seamless integration with Azure Batch job, task and file concepts along with
    full pass-through of the
    Azure Batch API
    to containers executed on compute nodes
  • Support for Azure Batch task dependencies
    allowing complex processing pipelines and DAGs
  • Support for merge or final task specification that automatically depends
    on all other tasks within the job
  • Support for job schedules and recurrences for automatic execution of
    tasks at set intervals
  • Support for live job and job schedule migration between pools
  • Support for Low Priority Compute Nodes
  • Support for deploying Batch compute nodes into a specified
    Virtual Network
    and pre-defined public IP addresses
  • Automatic setup of SSH or RDP users to all nodes in the compute pool and
    optional creation of SSH tunneling scripts to Docker Hosts on compute nodes
  • Support for custom host images
    including Shared Image Gallery
  • Support for Windows Containers
    on compliant Windows compute node pools with the ability to activate
    Azure Hybrid Use Benefit
    if applicable

Installation

Local Installation

Please see the installation guide
for more information regarding the various local installation options and
requirements.

Azure Cloud Shell

Batch Shipyard is integrated directly into
Azure Cloud Shell
and you can execute any Batch Shipyard workload using your web browser or
the Microsoft Azure Android
and iOS
app.

Simply request a Cloud Shell session and type shipyard to invoke the CLI;
no installation is required. Try Batch Shipyard now
in your browser.

Documentation and Recipes

Please refer to the
Batch Shipyard Documentation on Read the Docs.

Visit the
Batch Shipyard Recipes
section for various sample container workloads using Azure Batch and Batch
Shipyard.

Batch Shipyard Compute Node Host OS Support

Batch Shipyard is currently compatible with popular Azure Batch supported
Marketplace Linux VMs,
compliant Linux custom images,
and native Azure Batch
Windows Server with Containers
VMs. Please see the
platform image support
documentation for more information specific to Batch Shipyard support of
compute node host operating systems.

Change Log

Please see the
Change Log
for project history.


Please see this project’s Code of Conduct and
Contributing guidelines.