experimental/cpu/: instructlab-0.14.1 metadata and description

Simple index

CLI for interacting with InstructLab

author_email TBD <jon@example.com>
  • Development Status :: 3 - Alpha
  • Environment :: Console
  • License :: OSI Approved :: Apache Software License
  • License :: OSI Approved :: MIT License
  • Operating System :: MacOS :: MacOS X
  • Operating System :: POSIX :: Linux
  • Topic :: Scientific/Engineering :: Artificial Intelligence
  • Programming Language :: Python :: 3
  • Programming Language :: Python :: 3.9
  • Programming Language :: Python :: 3.10
  • Programming Language :: Python :: 3.11
  • Programming Language :: Python :: 3.12
  • Programming Language :: Python :: Implementation :: CPython
description_content_type text/markdown
license Apache-2.0 AND MIT
  • homepage, https://instructlab.io
  • source, https://github.com/instructlab/instructlab
  • issues, https://github.com/instructlab/instructlab/issues
  • click <9.0.0,>=8.1.7
  • click-didyoumean <0.4.0,>=0.3.0
  • datasets <3.0.0,>=2.18.0
  • gguf <0.7.0,>=0.6.0
  • GitPython <4.0.0,>=3.1.42
  • httpx
  • jsonschema <5.0.0,>=4.21.1
  • llama-cpp-python[server] ==0.2.55
  • numpy <2.0.0,>=1.26.4
  • openai <2.0.0,>=1.13.3
  • peft <0.10.0,>=0.9.0
  • prompt-toolkit <4.0.0,>=3.0.38
  • pydantic
  • pydantic-yaml
  • PyYAML <7.0.0,>=6.0.1
  • rich <14.0.0,>=13.3.1
  • rouge-score <0.2.0,>=0.1.2
  • sentencepiece <0.3.0,>=0.2.0
  • tokenizers <0.16.0,>=0.15.2
  • toml <0.11.0,>=0.10.2
  • torch <3.0.0,>=2.2.1
  • tqdm <5.0.0,>=4.66.2
  • transformers <=4.38.2,>=4.30.0
  • trl <0.8.0,>=0.7.11
  • wandb <0.17.0,>=0.16.4
  • langchain-text-splitters
  • yamllint <1.36.0,>=1.35.1
  • mlx <0.6.0,>=0.5.1 ; sys_platform == "darwin" and platform_machine == "arm64"
requires_python >=3.9
File Tox results History
682 KB
Python Wheel

InstructLab ๐Ÿถ (ilab)

Lint Tests Build Demo License

๐Ÿ“– Contents

Welcome to the InstructLab CLI

InstructLab ๐Ÿถ uses a novel synthetic data-based alignment tuning method for Large Language Models (LLMs.) The "lab" in InstructLab ๐Ÿถ stands for Large-Scale Alignment for ChatBots [1].

[1] Shivchander Sudalairaj*, Abhishek Bhandwaldar*, Aldo Pareja*, Kai Xu, David D. Cox, Akash Srivastava*. "LAB: Large-Scale Alignment for ChatBots", arXiv preprint arXiv: 2403.01081, 2024. (* denotes equal contributions)

โ“ What is ilab

ilab is a Command-Line Interface (CLI) tool that allows you to:

  1. Download a pre-trained Large Language Model (LLM).
  2. Chat with the LLM.

To add new knowledge and skills to the pre-trained LLM you have to add new information to the companion taxonomy repository. After that is done, you can:

  1. Use ilab to generate new synthetic training data based on the changes in your local taxonomy repository.
  2. Re-train the LLM with the new training data.
  3. Chat with the re-trained LLM to see the results.

The full process is described graphically in the workflow diagram.

[!IMPORTANT] It is important to understand that running InstructLab on a laptop will give you a low-fidelity approximation of both synthetic data generation (using the ilab generate command) and model instruction tuning (using the ilab train command, which uses QLoRA.) The quality of the results you get using these tools on a laptop will not be as high-fidelity as they might be using a larger teacher model and a different training method. We have optimized InstructLab to enable community members with modest hardware to be able to use the technique. If you have more sophisticated hardware, you can configure InstructLab to use a larger teacher model such as Mixtral in order to achieve higher-fidelity results.

๐Ÿ“‹ Requirements

NOTE: PyTorch 2.2.1 does not support torch.compile with Python 3.12. On Fedora 39+, install python3.11-devel and create the virtual env with python3.11 if you wish to use PyTorch's JIT compiler.

โœ… Getting started

๐Ÿงฐ Installing ilab

  1. If you are on Fedora Linux, install C++, Python 3.9+, and other necessary tools by running the following command:

    sudo dnf install g++ gcc make pip python3 python3-devel python3-GitPython

    Optional: If g++ is not found, try gcc-c++ by running the following command:

    sudo dnf install gcc-c++ gcc make pip python3 python3-devel python3-GitPython
  2. Create a new directory called instructlab to store the files the ilab CLI needs when running and cd into the directory by running the following command:

    mkdir instructlab
    cd instructlab

    NOTE: The following steps in this document use Python venv for virtual environments. However, if you use another tool such as pyenv or Conda Miniforge for managing Python environments on your machine continue to use that tool instead. Otherwise, you may have issues with packages that are installed but not found in venv.

  3. Install and activate your venv environment by running the following command:

    python3 -m venv venv
    source venv/bin/activate
    pip install git+https://github.com/instructlab/instructlab.git@stable

    NOTE: โณ pip install may take some time, depending on your internet connection.

  4. From your venv environment, verify ilab is installed correctly, by running the ilab command.


    Example output

    (venv) $ ilab
    Usage: ilab [OPTIONS] COMMAND [ARGS]...
    CLI for interacting with InstructLab.
    If this is your first time running InstructLab, it's best to start with `ilab init` to create the environment.
    --config PATH  Path to a configuration file.  [default: config.yaml]
    --version      Show the version and exit.
    --help         Show this message and exit.
    chat      Run a chat using the modified model
    check     (Deprecated) Check that taxonomy is valid
    convert   Converts model to GGUF
    diff      Lists taxonomy files that have changed since <taxonomy-base>...
    download  Download the model(s) to train
    generate  Generates synthetic data to enhance your example data
    init      Initializes environment for InstructLab
    list      (Deprecated) Lists taxonomy files that have changed since <taxonomy-base>.
    serve     Start a local server
    test      Runs basic test to ensure model correctness
    train     Takes synthetic data generated locally with `ilab generate`...

    IMPORTANT: every ilab command needs to be run from within your Python virtual environment. To enter the Python environment, run the following command:

    source venv/bin/activate

๐Ÿ—๏ธ Initialize ilab

  1. Initialize ilab by running the following command:

    ilab init

    Example output

    Welcome to InstructLab CLI. This guide will help you set up your environment.
    Please provide the following values to initiate the environment [press Enter for defaults]:
    Path to taxonomy repo [taxonomy]: <ENTER>
  2. When prompted by the interface, press Enter to add a new default config.yaml file.

  3. When prompted, clone the https://github.com/instructlab/taxonomy.git repository into the current directory by typing y.

    Optional: If you want to point to an existing local clone of the taxonomy repository, you can pass the path interactively or alternatively with the --taxonomy-path flag.

    Example output

    (venv) $ ilab init
    Welcome to InstructLab CLI. This guide will help you set up your environment.
    Please provide the following values to initiate the environment [press Enter for defaults]:
    Path to taxonomy repo [taxonomy]: <ENTER>
    `taxonomy` seems to not exists or is empty. Should I clone https://github.com/instructlab/taxonomy.git for you? [y/N]: y
    Cloning https://github.com/instructlab/taxonomy.git...
    Generating `config.yaml` in the current directory...
    Initialization completed successfully, you're ready to start using `lab`. Enjoy!

    ilab will use the default configuration file unless otherwise specified. You can override this behavior with the --config parameter for any ilab command.

    The taxonomy repository uses submodules to incorporate the taxonomy schema. When the ilab init command clones the taxonomy repository, it automatically handles the submodules. If you clone the taxonomy repository yourself, be sure to use the --recurse-submodules option on the git clone command and the git pull command when pulling updates from the remote repository. For example:

    git clone --recurse-submodules https://github.com/instructlab/taxonomy.git
    git pull --recurse-submodules

๐Ÿ“ฅ Download the model

๐Ÿด Serving the model

๐Ÿ“ฃ Chat with the model (Optional)

Because you're serving the model in one terminal window, you will have to create a new window and re-activate your Python virtual environment to run ilab chat command:

source venv/bin/activate
ilab chat

Before you start adding new skills and knowledge to your model, you can check its baseline performance by asking it a question such as what is the capital of Canada?.

NOTE: the model needs to be trained with the generated synthetic data to use the new skills or knowledge

(venv) $ ilab chat
โ•ญโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ system โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฎ
โ”‚ Welcome to InstructLab Chat w/ GGML-MERLINITE-7B-lab-Q4_K_M (type /h for help)                                                                                                                                                                    โ”‚
>>b> what is the capital of Canada                                                                                                                                                                                                 [S][default]
โ•ญโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ ggml-merlinite-7b-lab-Q4_K_M โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฎ
โ”‚ The capital city of Canada is Ottawa. It is located in the province of Ontario, on the southern banks of the Ottawa River in the eastern portion of southern Ontario. The city serves as the political center for Canada, as it is home to โ”‚
โ”‚ Parliament Hill, which houses the House of Commons, Senate, Supreme Court, and Cabinet of Canada. Ottawa has a rich history and cultural significance, making it an essential part of Canada's identity.                                   โ”‚
โ•ฐโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ elapsed 12.008 seconds โ”€โ•ฏ
>>>                                                                                                                                                                                                                               [S][default]

๐Ÿ’ป Creating new knowledge or skills and training the model

๐ŸŽ Contribute knowledge or compositional skills

Contribute new knowledge or compositional skills to your local taxonomy repository.

Detailed contribution instructions can be found in the taxonomy repository.

[!IMPORTANT] There is a limit to how much content can exist in the question/answer pairs for the model to process. Due to this, only add a maximum of around 2300 words to your question and answer seed example pairs in the qna.yaml file.

๐Ÿ“œ List and validate your new data

  1. List your new data by running the following command:

    ilab diff
  2. To ensure ilab is registering your new knowledge or skills, you can run the ilab diff command. The following is the expected result after adding the new compositional skill foo-lang:

    (venv) $ ilab diff
    Taxonomy in /taxonomy/ is valid :)

๐Ÿš€ Generate a synthetic dataset

  1. To generate a synthetic dataset based on your newly added knowledge or skill set in taxonomy repository, run the following command:

    ilab generate

    NOTE: โณ This can take from 15 minutes to 1+ hours to complete, depending on your computing resources.

    Example output

    (venv) $ ilab generate
    INFO 2024-02-29 19:09:48,804 lab.py:250 Generating model 'ggml-merlinite-7b-lab-Q4_K_M' using 10 CPUs,
    taxonomy: '/home/username/instructlab/taxonomy' and seed 'seed_tasks.json'
    0%|##########| 0/100 Cannot find prompt.txt. Using default prompt.
    98%|##########| 98/100 INFO 2024-02-29 20:49:27,582 generate_data.py:428 Generation took 5978.78s

    The synthetic data set will be three files in the newly created generated directory named generated*.json, test*.jsonl, and train*.jsonl.

[!NOTE] If you want to pickup from where a failed or canceled ilab generate left off, you can copy the generated*.json file into a file named regen.json. regen.json will be picked up at the start of lab generate when available. You should remove it when the process is completed.

  1. Verify the files have been created by running the ls generated command.

    (venv) $ ls generated/
    'generated_ggml-merlinite-7b-lab-0226-Q4_K_M_2024-02-29T19 09 48.json'   'train_ggml-merlinite-7b-lab-0226-Q4_K_M_2024-02-29T19 09 48.jsonl'
    'test_ggml-merlinite-7b-lab-0226-Q4_K_M_2024-02-29T19 09 48.jsonl'

    Optional: It is also possible to run the generate step against a different model via an OpenAI-compatible API. For example, the one spawned by ilab serve or any remote or locally hosted LLM (e.g. via ollama, LM Studio, etc.). Run the following command:

    ilab generate --endpoint-url http://localhost:8000/v1

๐Ÿ‘ฉโ€๐Ÿซ Train the model

There are three options to train the model on your synthetic data-enhanced dataset.

Note: Every ilab command needs to be run from within your Python virtual environment.

Train the model locally on Linux

ilab train

NOTE: โณ This step can potentially take several hours to complete depending on your computing resources.

ilab train outputs a brand-new model that can be served in the models directory called ggml-model-f16.gguf.

 (venv) $ ls models
 ggml-merlinite-7b-lab-Q4_K_M.gguf  ggml-model-f16.gguf

NOTE: ilab train ships with experimental support for GPU acceleration with Nvidia CUDA or AMD ROCm. See the GPU acceleration documentation for more details.

Train the model locally on an M-series Mac

To train the model locally on your M-Series Mac is as easy as running:

ilab train

Note: โณ This process will take a little while to complete (time can vary based on hardware and output of ilab generate but on the order of 5 to 15 minutes)

ilab train outputs a brand-new model that is saved in the <model_name>-mlx-q directory called adapters.npz (in Numpy compressed array format). For example:

(venv) $ ls instructlab-merlinite-7b-lab-mlx-q
adapters-010.npz        adapters-050.npz        adapters-090.npz        config.json             tokenizer.model
adapters-020.npz        adapters-060.npz        adapters-100.npz        model.safetensors       tokenizer_config.json
adapters-030.npz        adapters-070.npz        adapters.npz            special_tokens_map.json
adapters-040.npz        adapters-080.npz        added_tokens.json       tokenizer.jso

Training the model in the cloud

Follow the instructions in Training.

โณ Approximate amount of time taken on each platform:

After that's done, you can play with your model directly in the Google Colab or Kaggle notebook. Model trained on the cloud will be saved on the cloud. The model can also be downloaded and served locally.

๐Ÿ“œ Test the newly trained model

๐Ÿด Serve the newly trained model

  1. Stop the server you have running by entering ctrl+c keys in the terminal running the server.


    • ๐ŸŽ This step is only implemented for macOS with M-series chips (for now).

    • Before serving the newly trained model you must convert it to work with the ilab cli. The ilab convert command converts the new model into quantized GGUF format which is required by the server to host the model in the ilab serve command.

  2. Convert the newly trained model by running the following command:

    ilab convert
  3. Serve the newly trained model locally via ilab serve command with the --model argument to specify your new model:

    ilab serve --model-path <New model name>

    Which model should you select to serve? After running the ilab convert command, a few files and directories are generated. The one you will want to serve will end in .gguf and will exist in a directory with the suffix fused-pt. For example: instructlab-merlinite-7b-lab-mlx-q-fused-pt/ggml-model-Q4_K_M.gguf

๐Ÿ“ฃ Chat with the new model (not optional this time)

๐ŸŽ Submit your new knowledge or skills

Of course, the final step is, if you've improved the model, to open a pull-request in the taxonomy repository that includes the files (e.g. qna.yaml) with your improved data.

๐Ÿ“ฌ Contributing

Check out our contributing guide to learn how to contribute.