thinking_effort_processor

Thinking Effort Processor

The Thinking Effort Processor is an experimental tool designed to control the reasoning depth of large language models by manipulating explicit thinking tokens. This repository provides a framework to dynamically adjust how much "thinking" a language model performs during text generation.

Key Features:

• Dynamic Control: Adjust the intensity of reasoning through the scale_factor parameter.
• Compatibility: Works with models trained with explicit reasoning patterns, such as those using <think> and </think> tokens.
• Examples Provided: Includes various examples demonstrating different use cases, such as the bouncing ball example.

Benefits:

• No Retraining Required: Control reasoning depth without the need for retraining the model.
• Flexible Scaling: Easily adjust the thinking effort from low to high, allowing for tailored responses based on application needs.

Highlights:

• Supports integration with popular libraries like transformers and llama-cpp-python.
• Provides clear instructions for installation and usage, making it accessible for developers.

Explore the repository to see how you can implement and experiment with reasoning depth in your language models!