-
Notifications
You must be signed in to change notification settings - Fork 0
/
README
136 lines (98 loc) · 5.36 KB
/
README
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
FrameTime
=========
Measure the time your computer/application/monitor takes from receiving
a keypress to displaying something different.
Why?
----
I believe response time is hugely important for the _feel_ of software. To that
end I'm interested in measuring and comparing the response time of different
applications. In the past I've used typometer[1], Which is easy to use and
requires almost no setup.
Typometer has a pretty severe limitation however. It only measures the time
until another application can read the pixel, a value I ultimately don't care
about. Instead I want to know the entire roundtrip time. From me pressing the
key, to my eye being able to see the change on the monitor. No software only
solution can provide that.
Nvidia has the LDAT (Latency Display Analysis Tool) as part of their _reviewer
toolkit_[2], but to my knowledge only people with platforms large enough to
provide PR for Nvidia can get a hold of that. I'd have a hard time trusting
a measurement device I can't audit anyway.
FrameTime is an alternative to these options. It's cheap and available: You can
make it yourself for around $20. It's open: So you can audit and fix it. It's
external: So it captures end-to-end. It's composable: So you can automate it
and use it has part of your build pipeline.
[1]: https://pavelfatin.com/typometer/
[2]: https://www.nvidia.com/en-us/geforce/news/nvidia-reviewer-toolkit/
How it works
------------
The hardware emulates a USB HID keyboard and a serial device. The serial device
is used for control signal to configure the measurement and start it. When the
measurement begins the device starts a timer and sends a keystroke. Using
a photodiode the device continuously monitors the light level of the monitor
for some time while streaming the measurements and their timings over the
serial interface. When measurement is completed (after a set amount of time)
the device then sends a reset keystroke and emits a terminator on the serial
bus. The measurement can then be analyzed after the fact.
FrameTime consists of a firmware, which is flashed to the teensy, and a client,
which talks to the firmware and works with the results. The firmware is should
be compilable using the Makefile and the avr-gcc toolchain. to flash the
resulting hex file I use teensy-loader-cli as such:
teensy-loader-cli --mcu atmega32u4 main.hex
The teensy should then reboot and be ready. You will see that it will register
itself as an USB HIDevice.
The client is written in Python. To use it, place the device on the screen and
run the command
client.py -s <sample_count> -d <delay> -o data.measure
The sample_count is the number of tests you want to run, and the delay is the
time between tests. Measurements will be written to data.measure.
You can then analyze the results with
analyze.py data.measure --header
Which will output some interesting statistics. An example can be see in this
analysis data collected with xterm
title signal lag_min lag_delta rise_mean rise_stddev
xterm 19.106 43973.000 142593.000 62733.758 15776.126
analyse.py can take multiple measure files at once to output an analysis of all
of them.
Example
-------
We can compare the performance of cat on xterm and sublime text 3. We do this
by opening each application with a large font size and a black background.
For xterm this is accomplished with arguments
$ xterm -fa monaco -fs 100 -bg black
For sublime text we have to modify the settings. This is left as an extersize
to the reader.
We then position the measurement device such that the light sensor will recieve
more light after a press of the 'a' button. For xterm this means we place the
light sensor on top of the new cursor position. For sublime it means we place
it on top of the a letter. We then run the client
$ sleep 10; client.py -s 10000 -d 0.04 -o xterm.measure
$ sleep 10; client.py -s 10000 -d 0.04 -o subl.measure
The initial sleep is used to allow us to move focus back to the window we want
to test.
After both tests have run we analyze the results
$ analyze.py xterm:data3.measure sublime:subl.measure --header -t .01
title signal lag_min lag_delta rise_mean rise_stddev
xterm 19.106 48423.000 126472.000 62733.758 15776.126
sublime 16.358 97788.000 133648.000 52678.657 18754.689
The results show us that sublime text 3 seems to take 50 000 cycles more to
display our letter. Since the teensy runs at 16MHz we can convert it to seconds
$ units 50000/16Mhz
Definition: 0.003125 s
So sublime text 3 is about 3 milliseconds slower than xterm
Hardware
-------
The intended (and tested) hardware is a teensy2 with a SFH206K photodiode
connected between pin D4 and ground. Theoretically any atmega32u4 based
microprocessor should work.
Teensy 2.0 - $16 https://www.pjrc.com/store/teensy.html
SFH206K - $ 1.37 https://www2.mouser.com/productdetail/osram-opto-semiconductors/sfh-206-k?qs=K5ta8V%252bWhtZrKhv9kwxagw==
Enclosure - $ 1.00 Assumes you have a 3D printer handy
===================
Total - $18.37
Attribution
-----------
The USB portion of this code is largely cobbled together from the examples
provided by pjrc for the teensy, their copyright notice is included in the
LICENSE.pjrc file.
Original work is licensed under the GPLv3. A complete copy of the license can
be found in LICENSE.