minicons: flexible behavioral analyses of transformer LMs¶
minicons has two core functionalities:
Computing word and sentence level probability measures (e.g. per-word log-probability in context) using the scorer module.
Extracting contextual word and phrase level representations from transformer LMs across different layers using the cwe module, where
cwestands for contextual word embedding.
minicons can be an extremely handy tool to conduct large-scale behavioral analyses of models – it can handle any LM that is available in the huggingface model hub and perform efficient batched computations. Minicons runs both on CPUs and GPUs!
minicons also ships with two command-line binaries to explore single sentences or score large files of sentences! A detailed description will be added soon!
pip install minicons
Alternatively, if you would like to edit the package, make sure you have
poetry installed. You can grab it from here! Then:
1git clone email@example.com:kanishkamisra/minicons.git 2poetry shell # starts a new virtual environment for the package source. 3poetry install