Skip to content

Extract activations

On this page

  • You need: a model (pretrained or your own) and your images listed in a manifest, as in Set up and pick a model.
  • You get: one file with the activation of every chosen layer for every image, in manifest order, ready to compare with human data.

What is an activation?

When an image goes through a network, every layer turns the output of the previous layer into a new set of numbers. A convolutional layer produces a stack of feature maps (channels × height × width), and a fully connected layer produces a vector. These numbers, for one image, are that layer's activation pattern, the network's equivalent of a voxel pattern in an ROI.

We record them with thingsvision (Muttenthaler & Hebart, 2021). It loads models from torchvision, timm, CORnet, CLIP and more through one interface, runs your images through them, and returns the activations of the layers you ask for.

1. Load the model and find its layers

from pathlib import Path

import numpy as np
import pandas as pd
import torch
from thingsvision import get_extractor
from thingsvision.utils.data import DataLoader, ImageDataset
from torchvision import transforms

KIT = Path("dnn-toy-kit")
manifest = pd.read_csv(KIT / "manifest.csv")  # the image order every array will follow
device = "cuda" if torch.cuda.is_available() else "cpu"  # use the GPU when there is one

# Load ResNet-50 with its ImageNet weights; the extractor can record any of its layers
extractor = get_extractor(
    model_name="resnet50",
    source="torchvision",  # (1)!
    device=device,
    # Trained weights; False gives the same network with random weights
    pretrained=True,
    model_parameters={"weights": "IMAGENET1K_V2"},  # name the weights you report
)
# The top-level modules: the stem, four stages of residual blocks, and the head
print([name for name in extractor.get_module_names() if "." not in name])  # (2)!
  1. Other sources work the same way, for example source="timm" with any timm model name, or source="custom" with model_name="cornet-s", "Alexnet_ecoset", "clip" and others. The model list has them all.
  2. extractor.get_module_names() lists every module of the network by name, more than 150 for ResNet-50, because each stage contains several residual blocks. Names without a dot are the top level. print(extractor.show_model()) shows the whole network with every module.
Output 
['conv1', 'bn1', 'relu', 'maxpool', 'layer1', 'layer2', 'layer3', 'layer4', 'avgpool', 'fc']

We take the output of each stage (see Which modules to record below). maxpool is the end of the stem, and layer1 to layer4 are the four stages of residual blocks. avgpool averages each feature map over space and gives the last representation before the classifier.

2. Point it at your images

# Turn each sprite into the input ResNet-50 expects: 224 x 224 pixels, scaled like the
# ImageNet photos
preprocess = transforms.Compose([
    transforms.Resize(224, interpolation=transforms.InterpolationMode.NEAREST),  # (1)!
    transforms.ToTensor(),  # image -> tensor with values from 0 to 1
    # ImageNet mean and spread
    transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]),
])

# e.g. "cat_meadow_centre.png", in manifest order
file_names = [Path(f).name for f in manifest["file"]]
dataset = ImageDataset(
    root=str(KIT / "stimuli"),
    out_path="features",  # (2)!
    backend=extractor.get_backend(),
    file_names=file_names,  # (3)!
    transforms=preprocess,
)
# Stop if the order differs
assert Path("features/file_names.txt").read_text().split() == file_names
# 32 images per pass
batches = DataLoader(dataset, batch_size=32, backend=extractor.get_backend())
  1. For photographs, use the model's own preprocessing with transforms=extractor.get_transformations(). Here we enlarge the 48-pixel images to 224 pixels with nearest-neighbour interpolation, so they stay sharp.
  2. thingsvision writes the order in which it will read the images to features/file_names.txt. The assert checks it against the manifest.
  3. Listing the files explicitly makes thingsvision read them in manifest order. Without file_names it sorts them alphabetically. Give the names relative to root, and keep root a folder without subfolders.

3. Extract the layers

# Module names in the network (left) and the short names we use for them (right)
layers = ["maxpool", "layer1", "layer2", "layer3", "layer4", "avgpool"]  # the end of the stem, the four stages, the average
activations = extractor.extract_features(
    batches=batches,
    module_names=layers,  # (1)!
    # Keep the maps as channels x height x width, to plot them further down
    flatten_acts=False,
)

# One row per image: every value of every map, laid end to end
features = {name: activations[name].reshape(len(activations[name]), -1) for name in layers}  # (2)!
for name, x in features.items():
    print(f"{name:>5}: {x.shape}")  # images x features
  1. All layers come out of one pass through the network, as a dictionary from layer name to array.
  2. Each layer becomes one row per image, with the values of all its maps laid end to end. layer1 of ResNet-50 gives 256 × 56 × 56 = 802,816 numbers per image. RSA, decoding and encoding on the next pages use these full vectors, as Conwell et al. (2024) did for their RSA. Averaging each map down first, say to 6 × 6, would discard most of the spatial detail of the early stages, whose maps are 56 × 56. With thousands of images the full maps no longer fit in memory, and thingsvision can then write the features to disk as it extracts them (output_dir, see its low-memory options).

The output gives one matrix per layer, images × features:

maxpool: (144, 200704)
layer1: (144, 802816)
layer2: (144, 401408)
layer3: (144, 200704)
layer4: (144, 100352)
avgpool: (144, 2048)

Which modules to record

show_model() lists every module of the network, and most of them are steps inside what you would call a layer. In AlexNet, a convolutional layer is a convolution followed by a ReLU, and three of them end with max pooling. In a ResNet, convolutions, batch normalisation and ReLUs alternate inside residual blocks. What each kind of module gives you once the network is trained:

Module What it does when you run the model Record it?
Convolution (Conv2d) Filters its input; responses can be negative Usually take the ReLU after it: that is what the next layer receives
ReLU Sets negative responses to zero Yes, the usual output of a layer in a plain network such as AlexNet
Max pooling (MaxPool2d) Keeps the strongest response in each small patch Fine as the end of a stage: Brain-Score records AlexNet after conv1, conv2 and conv5 at the pooling
Batch normalisation (BatchNorm2d) Rescales and shifts each channel by fixed amounts, set during training Rarely: it is the convolution, rescaled channel by channel. Take the ReLU after it, or the block output
Dropout Passes its input through unchanged Never: it is identical to the module before it
Linear (fully connected) A weighted sum of all its inputs Take the ReLU after it (fc6, fc7). The last linear layer gives one score per training class: use it only if those classes are your question
Residual block (layer1.0, layer2, ...) Several of the steps above, plus the shortcut that adds the block's input The output of the whole block or stage, not the steps inside it (Schrimpf et al., 2018)
Global average pooling (avgpool) Averages each feature map over space The usual last representation of a ResNet, just before the classifier
Transformer block Attention and a small network, added to each token The output of each block. Take the class token or the average over tokens, and say which (see Other kinds of models below)

Whether to record a layer before or after its ReLU has not been settled by direct comparisons. Conwell et al. (2024) treat a convolution and the ReLU after it as two candidate layers and choose between them on independent data. Whatever you pick, use the same kind of module throughout, and name the modules in your methods. Which of the recorded layers to compare with a brain region is covered in Which layer to compare.

4. Save them with the image order

# Compressed: most values are zeros after the ReLU, so the file shrinks about sixfold
np.savez_compressed(
    "resnet50_features.npz",
    image_id=manifest["image_id"].to_numpy(dtype=str),  # (1)!
    **features,
)
  1. Saving the image names with the activations lets every later script check that rows line up with the human data, instead of trusting that the order never changed.

The file holds one array per layer, images × units, plus the image order:

Key Shape Dimensions
image_id (144,) images, the same order as manifest.csv
maxpool (144, 200704) images × units (64 channels × 56 × 56)
layer1 (144, 802816) images × units (256 channels × 56 × 56)
layer2 (144, 401408) images × units (512 channels × 28 × 28)
layer3 (144, 200704) images × units (1024 channels × 14 × 14)
layer4 (144, 100352) images × units (2048 channels × 7 × 7)
avgpool (144, 2048) images × units (2048 channels × 1 × 1)
Plot the feature maps of one image

The maps keep their shape in activations, so plotting the maps at the end of the stem for the cat takes a few lines:

import matplotlib.pyplot as plt

cat = manifest.index[manifest["image_id"] == "cat_meadow_centre"][0]  # the row of the cat on the meadow
fig, axes = plt.subplots(2, 8, figsize=(9, 2.6))  # the first 16 of the 64 channels
for channel, ax in enumerate(axes.flat):
    # white = 0, dark = strong response
    ax.imshow(activations["maxpool"][cat, channel], cmap="bone_r")
    ax.set_title(f"channel {channel}", fontsize=7)
    ax.set(xticks=[], yticks=[])
plt.show()

Your own trained model

A network you trained yourself (for example the ResNet-50 saved at the end of Train or fine-tune) goes through the same steps. Rebuild the architecture, load the weights, and wrap it with get_extractor_from_model:

from thingsvision import get_extractor_from_model
from torch import nn
from torchvision import models

model = models.resnet50()  # the architecture, with random weights for now
model.fc = nn.Linear(2048, 3)  # same head as in training: 3 categories
model.load_state_dict(torch.load("sprite_resnet50.pt", weights_only=True))  # (1)!
extractor = get_extractor_from_model(model=model, device=device, backend="pt")
  1. Copies the trained weights into the network. weights_only=True loads only the numbers, so a file from someone else cannot run code on your computer.

From here, sections 2 to 4 run unchanged.

Other kinds of models
  • Recurrent models (CORnet-RT, CORnet-S): a layer runs several times per image. Check in the thingsvision documentation which time step a module name refers to, and report it.
  • Vision transformers: layers output tokens (images × tokens × features). thingsvision can return the class token or the token average (model_parameters={"token_extraction": "cls_token"} or "avg_pool"). Say which in your methods.
  • Models thingsvision does not ship (VOneNet, TDANN, TopoNets): load them with their own package, then wrap them with get_extractor_from_model as above.

Where next?

  • Compare with a model

    Test which layers follow the scene, the position or the category of the images, with model RDMs and decoding.

  • Compare with human data

    Use resnet50_features.npz and the ROI patterns for RSA, decoding and encoding models.