Hyperparameter tuning with KerasTuner

Define a search space, run RandomSearch or Hyperband, persist results, and compare trials honestly instead of chasing the best-looking number.

Defining the search space

import keras_tuner as kt
import tensorflow as tf

def build_model(hp):
    model = tf.keras.Sequential()
    model.add(tf.keras.layers.Input(shape=(32,)))

    for i in range(hp.Int("n_layers", 1, 3)):
        model.add(tf.keras.layers.Dense(
            units=hp.Int(f"units_{i}", min_value=32, max_value=256, step=32),
            activation=hp.Choice(f"act_{i}", ["relu", "gelu"]),
        ))
        model.add(tf.keras.layers.Dropout(
            hp.Float(f"drop_{i}", 0.0, 0.5, step=0.1)))

    model.add(tf.keras.layers.Dense(1))

    model.compile(
        optimizer=tf.keras.optimizers.Adam(
            hp.Float("lr", 1e-4, 1e-2, sampling="log")),
        loss=tf.keras.losses.BinaryCrossentropy(from_logits=True),
        metrics=["accuracy"],
    )
    return model


tuner = kt.Hyperband(
    build_model,
    objective="val_accuracy",
    max_epochs=20,
    factor=3,
    directory="tuning",
    project_name="tabular_binary",
    overwrite=True,
)

• Int with a step, Float with sampling="log" for learning rates, Choice for a fixed set, Boolean for toggles.
• Names must be unique and stable. Using f"units_{i}" inside a loop lets the search vary depth without reusing a name.
• A log-uniform range for the learning rate is essential: linear sampling spends almost all trials at the high end of a decade.
• Keep the space small. Twelve hyperparameters with wide ranges will not be explored by a few dozen trials — pick the three or four that matter and fix the rest.

Running the search with callbacks

stop_early = tf.keras.callbacks.EarlyStopping(
    monitor="val_loss", patience=3, restore_best_weights=True)

tuner.search(
    X_train, y_train,
    epochs=20,
    validation_split=0.2,
    callbacks=[stop_early],
    verbose=1,
)

best = tuner.get_best_hyperparameters(num_trials=1)[0]
print(best.values)

# retrain from scratch on the full training data, with a real epoch budget
final = tuner.hypermodel.build(best)
final.fit(X_train, y_train, epochs=40, validation_split=0.1)

tuner.results_summary(num_trials=5)

Hyperband decides between trials on partial training curves, so it is only sound when the ranking is stable early. If a configuration that looks weak at epoch 3 becomes best at epoch 30, Hyperband will have already discarded it.

Reading the results honestly

import pandas as pd

results = pd.DataFrame([
    {"trial": t.trial_id, "score": t.score, "values": t.hyperparameters.values}
    for t in tuner.oracle.get_best_trials(num_trials=10)
])
print(results[["trial", "score"]].to_string(index=False))
print(results["values"].apply(lambda v: v.get("lr")).tolist())

# re-evaluate the top few on a split the tuner never saw
for t in tuner.oracle.get_best_trials(num_trials=3):
    model = tuner.hypermodel.build(t.hyperparameters)
    model.fit(X_train, y_train, epochs=30, verbose=0)
    score = model.evaluate(X_holdout, y_holdout, verbose=0)
    print(t.trial_id, score)

• The tuner's score is measured on its validation split, which was used to make selection decisions. It is now a slightly optimistic estimate, so confirm on a third split.
• A small difference between the top few trials is noise. Pick among them for speed, simplicity or parameter count instead of pretending the top score is meaningful.
• Record the seed, the tuner configuration and the search space alongside the results; a tuning run you cannot reproduce is a number you cannot trust.
• Tune the learning rate first, then architecture, then regularisation. Doing all three at once multiplies the number of trials by an order of magnitude.

FAQ

Related

TensorBoard and experiment tracking Regularisation and normalisation

Last refreshed 2026-09-18.