Skip to content

Commit

Permalink
Introduce rr.notebook_show() to simplify notebook experience (#5715)
Browse files Browse the repository at this point in the history
### What
As part of adding blueprint support to notebooks, I realized the
experience of creating and logging memory recordings felt very
incongruous with the normal rerun workflow.

I introduced a new mechanism `rr.notebook_show()`, which uses the
MemoryRecording behind the scenes, but doesn't require users to be aware
of it.

All Blueprint types now support `_repr_html_` and show the currently
active recording stream.

Uncovered several existing sharp-corners with notebooks:
- MemoryStream was still generating spurious warnings about dropped
messages, which I cleaned up.
- The notebook was using random for its identifiers, which, in the case
of examples was having its seed reset, leading to duplicate ids in the
DOM.

Lastly, updated the notebook cube example to use the new style:


![image](https://github.com/rerun-io/rerun/assets/3312232/cc9d354e-2744-4649-be2e-484499bc4f98)

### Checklist
* [x] I have read and agree to [Contributor
Guide](https://github.com/rerun-io/rerun/blob/main/CONTRIBUTING.md) and
the [Code of
Conduct](https://github.com/rerun-io/rerun/blob/main/CODE_OF_CONDUCT.md)
* [x] I've included a screenshot or gif (if applicable)
* [x] I have tested the web demo (if applicable):
* Using newly built examples:
[app.rerun.io](https://app.rerun.io/pr/5715/index.html)
* Using examples from latest `main` build:
[app.rerun.io](https://app.rerun.io/pr/5715/index.html?manifest_url=https://app.rerun.io/version/main/examples_manifest.json)
* Using full set of examples from `nightly` build:
[app.rerun.io](https://app.rerun.io/pr/5715/index.html?manifest_url=https://app.rerun.io/version/nightly/examples_manifest.json)
* [x] The PR title and labels are set such as to maximize their
usefulness for the next release's CHANGELOG
* [x] If applicable, add a new check to the [release
checklist](https://github.com/rerun-io/rerun/blob/main/tests/python/release_checklist)!

- [PR Build Summary](https://build.rerun.io/pr/5715)
- [Docs
preview](https://rerun.io/preview/c0177a9fb9cb1ef76f2eeba7b65aa7c861da2642/docs)
<!--DOCS-PREVIEW-->
- [Examples
preview](https://rerun.io/preview/c0177a9fb9cb1ef76f2eeba7b65aa7c861da2642/examples)
<!--EXAMPLES-PREVIEW-->
- [Recent benchmark results](https://build.rerun.io/graphs/crates.html)
- [Wasm size tracking](https://build.rerun.io/graphs/sizes.html)
  • Loading branch information
jleibs authored Mar 28, 2024
1 parent 34e3fec commit 8c0d389
Show file tree
Hide file tree
Showing 11 changed files with 441 additions and 182 deletions.
7 changes: 4 additions & 3 deletions .typos.toml
Original file line number Diff line number Diff line change
Expand Up @@ -137,8 +137,9 @@ zeroterminated = "null-terminated"
zero-terminated = "null-terminated"

[default]
# Work around for typos inside of 8-character hashes. These show up inside of ipynb.
# Work around for typos inside of hashes. These show up inside of ipynb.
# e.g. "f4e1caf9" -> `caf` should be `calf`
# Specifically limit ourselves to exactly 8 chars in a quoted strong.
# Specifically limit ourselves to exactly 8 chars in a quoted string, or
# 16 character hashses following a leading underscore.
# Just don't spell "defaced" wrong.
extend-ignore-re = ["\"[a-f0-9]{8}\""]
extend-ignore-re = ["\"[a-f0-9]{8}\"", "_[a-f0-9]{16}"]
59 changes: 34 additions & 25 deletions crates/re_sdk/src/log_sink.rs
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
use std::fmt;
use std::sync::Arc;

use parking_lot::RwLock;
use parking_lot::Mutex;
use re_log_types::{BlueprintActivationCommand, LogMsg, StoreId};

/// Where the SDK sends its log messages.
Expand Down Expand Up @@ -150,34 +150,44 @@ impl LogSink for MemorySink {

#[inline]
fn flush_blocking(&self) {}

#[inline]
fn drain_backlog(&self) -> Vec<LogMsg> {
self.0.take()
}
}

impl fmt::Debug for MemorySink {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
write!(
f,
"MemorySink {{ {} messages }}",
self.buffer().read().len()
)
write!(f, "MemorySink {{ {} messages }}", self.buffer().num_msgs())
}
}

#[derive(Default)]
struct MemorySinkStorageInner {
msgs: Vec<LogMsg>,
has_been_used: bool,
}

/// The storage used by [`MemorySink`].
#[derive(Default, Clone)]
pub struct MemorySinkStorage {
msgs: Arc<RwLock<Vec<LogMsg>>>,
inner: Arc<Mutex<MemorySinkStorageInner>>,
pub(crate) rec: Option<crate::RecordingStream>,
}

impl Drop for MemorySinkStorage {
fn drop(&mut self) {
for msg in self.msgs.read().iter() {
// Sinks intentionally end up with pending SetStoreInfo messages
// these are fine to drop safely. Anything else should produce a
// warning.
if !matches!(msg, LogMsg::SetStoreInfo(_)) {
re_log::warn!("Dropping data in MemorySink");
return;
let inner = self.inner.lock();
if !inner.has_been_used {
for msg in &inner.msgs {
// Sinks intentionally end up with pending SetStoreInfo messages
// these are fine to drop safely. Anything else should produce a
// warning.
if !matches!(msg, LogMsg::SetStoreInfo(_)) {
re_log::warn!("Dropping data in MemorySink");
return;
}
}
}
}
Expand All @@ -186,20 +196,16 @@ impl Drop for MemorySinkStorage {
impl MemorySinkStorage {
/// Write access to the inner array of [`LogMsg`].
#[inline]
fn write(&self) -> parking_lot::RwLockWriteGuard<'_, Vec<LogMsg>> {
self.msgs.write()
}

/// Read access to the inner array of [`LogMsg`].
#[inline]
pub fn read(&self) -> parking_lot::RwLockReadGuard<'_, Vec<LogMsg>> {
self.msgs.read()
fn write(&self) -> parking_lot::MappedMutexGuard<'_, Vec<LogMsg>> {
let mut inner = self.inner.lock();
inner.has_been_used = false;
parking_lot::MutexGuard::map(inner, |inner| &mut inner.msgs)
}

/// How many messages are currently written to this memory sink
#[inline]
pub fn num_msgs(&self) -> usize {
self.read().len()
self.inner.lock().msgs.len()
}

/// Consumes and returns the inner array of [`LogMsg`].
Expand All @@ -212,7 +218,7 @@ impl MemorySinkStorage {
// in this flush; it's just a matter of making the table batcher tick early.
rec.flush_blocking();
}
std::mem::take(&mut *self.msgs.write())
std::mem::take(&mut (self.write()))
}

/// Convert the stored messages into an in-memory Rerun log file.
Expand All @@ -227,7 +233,10 @@ impl MemorySinkStorage {
let mut encoder =
re_log_encoding::encoder::Encoder::new(encoding_options, &mut buffer)?;
for sink in sinks {
for message in sink.read().iter() {
let mut inner = sink.inner.lock();
inner.has_been_used = true;

for message in &inner.msgs {
encoder.append(message)?;
}
}
Expand Down
30 changes: 23 additions & 7 deletions examples/python/notebook/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,26 +4,42 @@ Rerun has limited support for direct embedding within a [Jupyter](https://jupyte
Many additional environments beyond Jupyter are supported such as [Google Colab](https://colab.research.google.com/)
or [VSCode](https://code.visualstudio.com/blogs/2021/08/05/notebooks).

In order to show a rerun viewer inline within the notebook you need to use a special in-memory
recording:
```
rec = rr.memory_recording()
In order to show a rerun viewer inline within the notebook, you can call:

```python
rr.init("rerun_example_notebook")

rr.log(...)

rr.notebook_show()
```

After creating this recording all the normal rerun commands will work as expected and log
to this recording instance. When you are ready to show it you can return it at the end of your cell
or call `rec.show()`.
This will show the contents of the current global recording stream. Note that the global stream will accumulate
data in-memory. You can reset the stream by calling `rr.init` again to establish a new global context.

As with the other stream viewing APIs (`rr.show`, `rr.connect`, `rr.spawn`), you can alternatively pass
a specific recording instance to `notebook_show`

```python
rec = rr.new_recording("rerun_example_notebook_local")

rec.log(...)

rr.notebook_show(recording=rec)
```

# Running in Jupyter

The easiest way to get a feel for working with notebooks is to use it:

Install jupyter

```
pip install -r requirements.txt
```

Open the notebook

```
jupyter notebook cube.ipynb
```
Expand Down
Loading

0 comments on commit 8c0d389

Please sign in to comment.