项目作者: larq

项目描述 :
A small library for managing deep learning models, hyperparameters and datasets
高级语言: Python
项目地址: git://github.com/larq/zookeeper.git
创建时间: 2019-05-02T09:32:55Z
项目社区:https://github.com/larq/zookeeper

开源协议:Apache License 2.0

下载


Zookeeper

GitHub Actions Codecov PyPI - Python Version PyPI PyPI - License Code style: black

A small library for configuring modular applications.

Installation

  1. pip install zookeeper

Components

The fundamental building blocks of Zookeeper are components. The
@component decorator is used to turn classes into
components. These component classes can have configurable fields, which are
declared with the Field constructor and class-level type annotations. Fields
can be created with or without default values. Components can also be nested,
with ComponentFields, such that child componenents can access the field values
defined on their parents.

For example:

  1. from zookeeper import component
  2. @component
  3. class ChildComponent:
  4. a: int = Field() # An `int` field with no default set
  5. b: str = Field("foo") # A `str` field with default value `"foo"`
  6. @component
  7. class ParentComponent:
  8. a: int = Field() # The same `int` field as the child
  9. child: ChildComponent = ComponentField() # A nested component field, of type `ChildComponent`

After instantiation, components can be ‘configured’ with a configuration
dictionary, containing values for a tree of nested fields. This process
automatically injects the correct values into each field.

If a child sub-component declares a field which already exists in some
containing ancestor component, then it will pick up the value that’s set on the
parent, unless a ‘scoped’ value is set on the child.

For example:

  1. from zookeeper import configure
  2. p = ParentComponent()
  3. configure(
  4. p,
  5. {
  6. "a": 5,
  7. "child.a": 4,
  8. }
  9. )
  10. >>> 'ChildComponent' is the only concrete component class that satisfies the type
  11. >>> of the annotated parameter 'ParentComponent.child'. Using an instance of this
  12. >>> class by default.
  13. print(p)
  14. >>> ParentComponent(
  15. >>> a = 5,
  16. >>> child = ChildComponent(
  17. >>> a = 4,
  18. >>> b = "foo"
  19. >>> )
  20. >>> )

Tasks and the CLI

The @task decorator is used to define Zookeeper tasks and
can be applied to any class that implements an argument-less run method. Such
tasks can be run through the Zookeeper CLI, with parameter values passed in
through CLI arguments (configure is implicitly called).

For example:

  1. from zookeeper import cli, task
  2. @task
  3. class UseChildA:
  4. parent: ParentComponent = ComponentField()
  5. def run(self):
  6. print(self.parent.child.a)
  7. @task
  8. class UseParentA(UseChildA):
  9. def run(self):
  10. print(self.parent.a)
  11. if __name__ == "__main__":
  12. cli()

Running the above file then gives a nice CLI interface:

  1. python test.py use_child_a
  2. >>> ValueError: No configuration value found for annotated parameter 'UseChildA.parent.a' of type 'int'.
  3. python test.py use_child_a a=5
  4. >>> 5
  5. python test.py use_child_a a=5 child.a=3
  6. >>> 3
  7. python test.py use_parent_a a=5 child.a=3
  8. >>> 5

Using Zookeeper to define Larq or Keras experiments

See examples/larq_experiment.py for an example of
how to use Zookeeper to define all the necessary components (dataset,
preprocessing, and model) of a Larq experiment: training a BinaryNet on
MNIST. This example can be easily adapted to other Larq or Keras models and
other datasets.