minicons: flexible behavioral analyses of transformer LMs

Contents:

Introduction

minicons has two core functionalities:

  • Computing word and sentence level probability measures (e.g. per-word log-probability in context) using the scorer module.

  • Extracting contextual word and phrase level representations from transformer LMs across different layers using the cwe module, where cwe stands for contextual word embedding.

In general, minicons can be an extremely handy tool to conduct large-scale behavioral analyses of models – it can handle any LM that is available in the huggingface model hub and perform efficient batched computations. Minicons runs both on CPUs and GPUs!

minicons also ships with two command-line binaries to explore single sentences or score large files of sentences! A detailed description will be added soon!

Getting Started

Install minicons using pip:

pip install minicons

Alternatively, if you would like to edit the package, make sure you have poetry installed. You can grab it from here! Then:

1git clone git@github.com:kanishkamisra/minicons.git
2poetry shell # starts a new virtual environment for the package source.
3poetry install

Examples

  1. Calculating surprisals from Language Models

  2. Extract word and phrase representations from Transformers

Minicons logo made by Flat Icons from flaticon.