pulpo

A library and sdk for non-contextual and contextual Multi-Armed-Bandit (MAB) algorithms for multiple use cases. The sdk version enables you to deploy it on AWS.

Installation

Prerequisites

pulpo requires Python 3.6+

Install pulpo

At the command line:

pip install pulpo

How To

Library

Pulpo can be used as a library to instantiate and run online MABs. The core of the library are the bandit implementations under pulpo/badnits module. The super class of them is OnlineBandit if you wish to implement your own MAB implementation.

Currently, the following MAB implementations are available:

Epsilon Greedy (EGreedy)

The usage of this library starts from Pulpo class. For example, let’s say that we want to run 1 online bandit with 3 arms using EGreedy, then:

from pulpo.bandits.dataclasses import EpsilonGreedyArm
from pulpo.bandits.epsilon_greedy import EGreedy
from pulpo.pulpo import Pulpo
arm_names = ["article1", "article2", "article3"]
arms = [EpsilonGreedyArm(name, 1, 0) for name in arm_names]  # priors for n=1 and sum=0, i.e. steps and total reward so far
bandit = EGreedy("article_recommendation", arms, epsilon=0.9)
pulpo = Pulpo([bandit])  # Instantiate Pulpo
arm_id = pulpo.choose(bandit.bandit_id)  # to get an arm decision
# get some feedback for the arm decision and pass it to back
feedback = 1.0
pulpo.update(bandit.bandit_id, arm_id, feedback)

Alternatively, the Pulpo class can be instantiated in the following manner:

import json
from pulpo.pulpo import Pulpo
bandit_id = "article_recommendation"
config = config = [{"bandit_id": bandit_id, "bandit_type": "epsilon_greedy", "arm_ids": ["article1", "article2", "article3"]}]
pulpo = Pulpo.make_from_json(json.dumps(config))  # Instantiate Pulpo
arm_id = pulpo.choose(bandit_id)  # to get an arm decision
# get some feedback for the arm decision and pass it to back
feedback = 1.0
pulpo.update(bandit_id, arm_id, feedback)

AWS SDK

Pulpo can be used as an sdk to deploy and run MABs on AWS. Soon…