OpenBox: Generalized and Efficient Blackbox Optimization System

OpenBox is an efficient open-source system designed for solving generalized black-box optimization (BBO) problems, such as automatic hyper-parameter tuning, automatic A/B testing, experimental design, database knob tuning, processor architecture and circuit design, resource allocation, automatic chemical design, etc.

The design of OpenBox follows the philosophy of providing “BBO as a service” - we opt to implement OpenBox as a distributed, fault-tolerant, scalable, and efficient service, with a wide range of application scope, stable performance across problems and advantages such as ease of use, portability, and zero maintenance.

There are two ways to use OpenBox: Standalone python package and Online BBO service.


Who should consider using OpenBox

  • Those who want to tune hyper-parameters for their ML tasks automatically.

  • Those who want to find the optimal configuration for their configuration search tasks (e.g., database knob tuning).

  • Data platform owners who want to provide BBO service in their platform.

  • Researchers and data scientists who want to solve generalized BBO problems easily.


OpenBox capabilities

OpenBox has a wide range of functionality scope, which includes:

  1. BBO with multiple objectives and constraints.

  2. BBO with transfer learning.

  3. BBO with distributed parallelization.

  4. BBO with multi-fidelity acceleration.

  5. BBO with early stops.

In the following, we provide a taxonomy of existing BBO systems:

System/Package

Multi-obj.

FIOC

Constraint

History

Distributed

Hyperopt

×

×

×

Spearmint

×

×

×

×

SMAC3

×

×

×

×

BoTorch

×

×

×

GPflowOPT

×

×

×

Vizier

×

×

HyperMapper

×

×

HpBandSter

×

×

×

OpenBox

  • FIOC: Support different input variable types, including Float, Integer, Ordinal and Categorical.

  • Multi-obj.: Support optimizing multiple objectives.

  • Constraint: Support inequality constraints.

  • History: Support injecting prior knowledge from previous tasks into the current search. (△ means the system cannot support it for general cases)

  • Distributed: Support parallel evaluations in a distributed environment.


Installation

Please refer to our Installation Guide.


Quick Start

In the following, we provide an example of optimizing the Branin function. For more description of this example, please refer to Quick Start.

import numpy as np
from openbox import Optimizer, sp

# Define Search Space
space = sp.Space()
x1 = sp.Real("x1", -5, 10, default_value=0)
x2 = sp.Real("x2", 0, 15, default_value=0)
space.add_variables([x1, x2])

# Define Objective Function
def branin(config):
   x1, x2 = config['x1'], config['x2']
   y = (x2-5.1/(4*np.pi**2)*x1**2+5/np.pi*x1-6)**2+10*(1-1/(8*np.pi))*np.cos(x1)+10
   return y

# Run
if __name__ == '__main__':
   opt = Optimizer(branin, space, max_runs=50, task_id='quick_start')
   history = opt.run()
   print(history)

Documentation



Releases and Contributing

OpenBox has a frequent release cycle. Please let us know if you encounter a bug by filling an issue.

We appreciate all contributions. If you are planning to contribute any bug-fixes, please do so without further discussions.

If you plan to contribute new features, new modules, etc. please first open an issue or reuse an existing issue, and discuss the feature with us.

To learn more about making a contribution to OpenBox, please refer to our how-to-contribute page.

We appreciate all contributions and thank all the contributors!




Feedback


License

The entire codebase is under MIT license.