On this page
torch.utils.tensorboard
Before going further, more details on TensorBoard can be found at https://www.tensorflow.org/tensorboard/
Once you’ve installed TensorBoard, these utilities let you log PyTorch models and metrics into a directory for visualization within the TensorBoard UI. Scalars, images, histograms, graphs, and embedding visualizations are all supported for PyTorch models and tensors as well as Caffe2 nets and blobs.
The SummaryWriter class is your main entry to log data for consumption and visualization by TensorBoard. For example:
import torch
import torchvision
from torch.utils.tensorboard import SummaryWriter
from torchvision import datasets, transforms
# Writer will output to ./runs/ directory by default
writer = SummaryWriter()
transform = transforms.Compose([transforms.ToTensor(), transforms.Normalize((0.5,), (0.5,))])
trainset = datasets.MNIST('mnist_train', train=True, download=True, transform=transform)
trainloader = torch.utils.data.DataLoader(trainset, batch_size=64, shuffle=True)
model = torchvision.models.resnet50(False)
# Have ResNet model take in grayscale rather than RGB
model.conv1 = torch.nn.Conv2d(1, 64, kernel_size=7, stride=2, padding=3, bias=False)
images, labels = next(iter(trainloader))
grid = torchvision.utils.make_grid(images)
writer.add_image('images', grid, 0)
writer.add_graph(model, images)
writer.close()
This can then be visualized with TensorBoard, which should be installable and runnable with:
pip install tensorboard
tensorboard --logdir=runs
Lots of information can be logged for one experiment. To avoid cluttering the UI and have better result clustering, we can group plots by naming them hierarchically. For example, “Loss/train” and “Loss/test” will be grouped together, while “Accuracy/train” and “Accuracy/test” will be grouped separately in the TensorBoard interface.
from torch.utils.tensorboard import SummaryWriter
import numpy as np
writer = SummaryWriter()
for n_iter in range(100):
writer.add_scalar('Loss/train', np.random.random(), n_iter)
writer.add_scalar('Loss/test', np.random.random(), n_iter)
writer.add_scalar('Accuracy/train', np.random.random(), n_iter)
writer.add_scalar('Accuracy/test', np.random.random(), n_iter)
Expected result:

class torch.utils.tensorboard.writer.SummaryWriter(log_dir=None, comment='', purge_step=None, max_queue=10, flush_secs=120, filename_suffix='')
[source]-
Writes entries directly to event files in the log_dir to be consumed by TensorBoard.
The
SummaryWriter
class provides a high-level API to create an event file in a given directory and add summaries and events to it. The class updates the file contents asynchronously. This allows a training program to call methods to add data to the file directly from the training loop, without slowing down training.__init__(log_dir=None, comment='', purge_step=None, max_queue=10, flush_secs=120, filename_suffix='')
[source]-
Creates a
SummaryWriter
that will write out events and summaries to the event file.- Parameters
-
- log_dir (str) – Save directory location. Default is runs/CURRENT_DATETIME_HOSTNAME, which changes after each run. Use hierarchical folder structure to compare between runs easily. e.g. pass in ‘runs/exp1’, ‘runs/exp2’, etc. for each new experiment to compare across them.
- comment (str) – Comment log_dir suffix appended to the default
log_dir
. Iflog_dir
is assigned, this argument has no effect. - purge_step (int) – When logging crashes at step
and restarts at step
, any events whose global_step larger or equal to
will be purged and hidden from TensorBoard. Note that crashed and resumed experiments should have the same
log_dir
. - max_queue (int) – Size of the queue for pending events and summaries before one of the ‘add’ calls forces a flush to disk. Default is ten items.
- flush_secs (int) – How often, in seconds, to flush the pending events and summaries to disk. Default is every two minutes.
- filename_suffix (str) – Suffix added to all event filenames in the log_dir directory. More details on filename construction in tensorboard.summary.writer.event_file_writer.EventFileWriter.
Examples:
from torch.utils.tensorboard import SummaryWriter # create a summary writer with automatically generated folder name. writer = SummaryWriter() # folder location: runs/May04_22-14-54_s-MacBook-Pro.local/ # create a summary writer using the specified folder name. writer = SummaryWriter("my_experiment") # folder location: my_experiment # create a summary writer with comment appended. writer = SummaryWriter(comment="LR_0.1_BATCH_16") # folder location: runs/May04_22-14-54_s-MacBook-Pro.localLR_0.1_BATCH_16/
add_scalar(tag, scalar_value, global_step=None, walltime=None, new_style=False, double_precision=False)
[source]-
Add scalar data to summary.
- Parameters
-
- tag (str) – Data identifier
- scalar_value (float or string/blobname) – Value to save
- global_step (int) – Global step value to record
- walltime (float) – Optional override default walltime (time.time()) with seconds after epoch of event
- new_style (boolean) – Whether to use new style (tensor field) or old style (simple_value field). New style could lead to faster data loading.
Examples:
from torch.utils.tensorboard import SummaryWriter writer = SummaryWriter() x = range(100) for i in x: writer.add_scalar('y=2x', i * 2, i) writer.close()
Expected result:
add_scalars(main_tag, tag_scalar_dict, global_step=None, walltime=None)
[source]-
Adds many scalar data to summary.
- Parameters
Examples:
from torch.utils.tensorboard import SummaryWriter writer = SummaryWriter() r = 5 for i in range(100): writer.add_scalars('run_14h', {'xsinx':i*np.sin(i/r), 'xcosx':i*np.cos(i/r), 'tanx': np.tan(i/r)}, i) writer.close() # This call adds three values to the same scalar plot with the tag # 'run_14h' in TensorBoard's scalar section.
Expected result:
add_histogram(tag, values, global_step=None, bins='tensorflow', walltime=None, max_bins=None)
[source]-
Add histogram to summary.
- Parameters
-
- tag (str) – Data identifier
- values (torch.Tensor, numpy.ndarray, or string/blobname) – Values to build histogram
- global_step (int) – Global step value to record
- bins (str) – One of {‘tensorflow’,’auto’, ‘fd’, …}. This determines how the bins are made. You can find other options in: https://docs.scipy.org/doc/numpy/reference/generated/numpy.histogram.html
- walltime (float) – Optional override default walltime (time.time()) seconds after epoch of event
Examples:
from torch.utils.tensorboard import SummaryWriter import numpy as np writer = SummaryWriter() for i in range(10): x = np.random.random(1000) writer.add_histogram('distribution centers', x + i, i) writer.close()
Expected result:
add_image(tag, img_tensor, global_step=None, walltime=None, dataformats='CHW')
[source]-
Add image data to summary.
Note that this requires the
pillow
package.- Parameters
-
- tag (str) – Data identifier
- img_tensor (torch.Tensor, numpy.ndarray, or string/blobname) – Image data
- global_step (int) – Global step value to record
- walltime (float) – Optional override default walltime (time.time()) seconds after epoch of event
- dataformats (str) – Image data format specification of the form CHW, HWC, HW, WH, etc.
- Shape:
-
img_tensor: Default is . You can use
torchvision.utils.make_grid()
to convert a batch of tensor into 3xHxW format or calladd_images
and let us do the job. Tensor with , , is also suitable as long as correspondingdataformats
argument is passed, e.g.CHW
,HWC
,HW
.
Examples:
from torch.utils.tensorboard import SummaryWriter import numpy as np img = np.zeros((3, 100, 100)) img[0] = np.arange(0, 10000).reshape(100, 100) / 10000 img[1] = 1 - np.arange(0, 10000).reshape(100, 100) / 10000 img_HWC = np.zeros((100, 100, 3)) img_HWC[:, :, 0] = np.arange(0, 10000).reshape(100, 100) / 10000 img_HWC[:, :, 1] = 1 - np.arange(0, 10000).reshape(100, 100) / 10000 writer = SummaryWriter() writer.add_image('my_image', img, 0) # If you have non-default dimension setting, set the dataformats argument. writer.add_image('my_image_HWC', img_HWC, 0, dataformats='HWC') writer.close()
Expected result: