Skip to content

Pretty print #934

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Oct 7, 2024
Merged

Pretty print #934

merged 7 commits into from
Oct 7, 2024

Conversation

plaguss
Copy link
Contributor

@plaguss plaguss commented Aug 27, 2024

Description

This PR adds a base method to allow easily printing the prompts of the tasks.

from distilabel.steps.tasks import URIAL

# only importable if placed at the root at the repo, any LLM will work
from tests.unit.conftest import DummyLLM

t = URIAL(llm=DummyLLM())
t.load()
t.print()
...
╭────────────────────────────────────────────────────────────────── Prompt: URIAL  ──────────────────────────────────────────────────────────────────╮
│ ╭───────────────────────────────────────────────────────────────── User Message ─────────────────────────────────────────────────────────────────╮ │
│ │ # Instruction                                                                                                                                  │ │
│ │                                                                                                                                                │ │
│ │ Below is a list of conversations between a human and an AI assistant (you).                                                                    │ │
│ │ Users place their queries under "# User:", and your responses are under  "# Assistant:".                                                       │ │
│ │ You are a helpful, respectful, and honest assistant.                                                                                           │ │
│ │ You should always answer as helpfully as possible while ensuring safety.                                                                       │ │
│ │ Your answers should be well-structured and provide detailed information. They should also have an engaging tone.                               │ │
│ │ Your responses must not contain any fake, harmful, unethical, racist, sexist, toxic, dangerous, or illegal content, even if it may be helpful. │ │
│ │ Your response must be socially responsible, and thus you can refuse to answer some controversial topics.                                       │ │
│ │                                                                                                                                                │ │
│ │                                                                                                                                                │ │
│ │ # User:                                                                                                                                        │ │
│ │                                                                                                                                                │ │
│ │ <PLACEHOLDER_INSTRUCTION>                                                                                                                      │ │
│ │                                                                                                                                                │ │
│ │ # Assistant:                                                                                                                                   │ │
│ ╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯ │
╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

Example screenshot:

image

For new tasks, we should review the prompt, and if not rendering properly, override the _sample_inputs method to return the appropriate content.

Run the following to see all the prompts at once:

pytest tests/integration/test_prints.py -s -vv

@plaguss plaguss requested a review from gabrielmbmb August 27, 2024 19:18
@plaguss plaguss self-assigned this Aug 27, 2024
@plaguss plaguss added this to the 1.4.0 milestone Aug 27, 2024
@plaguss plaguss requested a review from dvsrepo August 27, 2024 19:19
Copy link

Documentation for this PR has been built. You can view it at: https://distilabel.argilla.io/pr-934/

Copy link

codspeed-hq bot commented Aug 27, 2024

CodSpeed Performance Report

Merging #934 will not alter performance

Comparing pretty-print (a213728) with develop (2ce44f0)

Summary

✅ 1 untouched benchmarks

@plaguss plaguss marked this pull request as ready for review August 28, 2024 07:51
@plaguss plaguss added enhancement New feature or request question Further information is requested labels Sep 23, 2024
@plaguss plaguss merged commit 87683f0 into develop Oct 7, 2024
7 checks passed
@plaguss plaguss deleted the pretty-print branch October 7, 2024 13:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request question Further information is requested
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant