Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

--track-energy feature #125

Open
wants to merge 13 commits into
base: main
Choose a base branch
from
Open

--track-energy feature #125

wants to merge 13 commits into from

Conversation

cappadokes
Copy link

Python is popular, even on resource-constrained devices. Energy is important. A nice-to-have feature would be a pyperf option that, if enabled, measures energy consumption instead of execution time. The only requirement for this pull request to bear meaning is the availability of some always-increasing measurement of the total energy spent by a device component. An example is the interface exposed by Intel's RAPL, which was used as the environment for this pull request.

Instructions for utilizing the --track-energy option are:

  1. Write a dummy C library containing a single function that just reads from a file saved at the location denoted by the function's input parameter (a sample implementation is included as libreaden.so, the corresponding source file being read_file.c). Save the library in pyperf/pyperf.
  2. Export the following environment variables:
    • LIBREADEN, containing the path to the C library.
    • ENFILE, containing the path to the energy measurement file.
  3. Run your pyperf script with --track-energy and --inherit-environ LIBREADEN,ENFILE.

Measurements saved will now be energy instead of execution time.

The output will mention "seconds" instead of Joules or whatever. We haven't worked on the measurement units metadata yet.

@@ -420,7 +423,7 @@ def _main(self, task):
if task.name in self._bench_names:
raise ValueError("duplicated benchmark name: %r" % task.name)
self._bench_names.add(task.name)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unnecessary spaces in an empty line.

Suggested change

@vstinner
Copy link
Member

Run your pyperf script with --track-energy and --inherit-environ LIBREADEN,ENFILE.

--track-energy should imply --inherit-environ LIBREADEN,ENFILE.

@vstinner
Copy link
Member

What is LIBREADEN? Where does it come from? Please write documentation.

@cappadokes
Copy link
Author

What is LIBREADEN? Where does it come from? Please write documentation.

I am sorry, this is my first open-source contribution ever. I tried to be instructive in this PR's initial text. What qualifies as 'documentation' exactly? Do you mean editing the repo's README? Something else?

@corona10
Copy link
Member

@cappadokes
I love this feature but same as @vstinner comment, Can you please leave a link related to LIBREADEN, it would help to review this PR :)

@cappadokes
Copy link
Author

cappadokes commented Mar 11, 2022

@cappadokes I love this feature but same as @vstinner comment, Can you please leave a link related to LIBREADEN, it would help to review this PR :)

OK, I will do my best to expand below. It would be great if you could then direct me on what specific places of this repo other than our thread here this information should exist.

The present PR introduces the implementation of a simple energy consumption benchmarking feature. It was developed and tested on an Intel(R) Core(TM) i7-8750H x86_64 Linux machine for pyperf 2.3.1, with CPython 3.10.2.

Requirements

  • A file-based interface exposing the system's aggregate energy consumption over time. Modern Intel CPUs provide such an interface via RAPL. On the example sysfs tree included in the previous link, you can see some files named energy_uj. These are single-line files containing an always-increasing value of the total microJoules consumed from the component since startup. There are many energy_uj files in many places of the intel-rapl tree, each one corresponding to a different scale of measurement (e.g. the whole package, the CPU, its caches, the DRAM).
  • A dummy C shared library (dummy since it does one thing only) for accessing the interface described above. This format was chosen so as to a) be as fast as possible, compared for instance to reading a file through Python and b) be agnostic with regard to the specific interface employed by the machine. An example implementation is read_file.c, which I include to this PR. Once the user has compiled such a library, she should save it in pyperf/pyperf (I used ctypes.CDLL to invoke it, yet placing the .so in arbitrary spots and calling its contents proved non-trivial for my skills).

Usage

export ENFILE=            #    Absolute path to energy_uj or equivalent resource
export LIBREADEN=    #    Absolute path to pyperf/pyperf/dummy_c_library.so
python3 pyperf_using_benchmark.py --track-energy --inherit-environ ENFILE,LIBREADEN

Motivation
I discovered pyperf through pyperformance, since I wanted to do some legit energy-related experiments on real-world benchmarks. For convenience (and since --inherit-environ is exposed yet broken there) I have also created a PR there in order to integrate it all together.

First experiences
I have produced sets of ~20 stable pyperformance benchmark results for my machine's CPU and DRAM energy consumption while using the --rigorous option. It's a small but non-negligible number compared to the ~60 total cases in pyperformance. As I'm writing this I realized that my experiments did not use the --system-tune feature, so a new batch might prove even more productive.

Limitation
The only thing not implemented yet is the measurement units metadata. Thus everything is currently reported as seconds instead of microJoules (the unit should however depend on the energy interface. So hard-coding microjoules is not the way to go). But dealing with this detail looks neither urgent nor challenging for now.

@vstinner
Copy link
Member

The documentation source lives in the doc/ subdirectory and is rendered at: https://pyperf.readthedocs.io/

@cappadokes
Copy link
Author

The documentation source lives in the doc/ subdirectory and is rendered at: https://pyperf.readthedocs.io/

And should I directly edit the .rst files in doc/ ? Are they a result of some automatic docs generation utility (and thus require that I edit the .py source code instead of the .rst's)? Again, sorry for the spamming, but dealing with these from the get-go will save time from all of us.

@cappadokes
Copy link
Author

Hi all. I have now updated the documentation so as to include --track-energy. I have started the analogous PR on the pyperformance side: python/pyperformance#140

There is nothing more to be done at this point.

Copy link
Member

@corona10 corona10 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cappadokes Sorry for the delay, I will take a look at this PR ASAP
cc @vstinner

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants