Battery

[carissalow]

Originally posted by @Meng6

Unify battery data for iOS client V1

We transform iOS (only iOS client V1 needs this step as V2 does it out-of-the-box) battery logs into Android's format. iOS stores battery status: 0=unknown, 1=unplugged, 2=charging, 3=full, as opposed to Android's event: 1=unknown, 2=charging, 3=discharging, 4=not charging, 5=full.

We follow this algorithm to convert iOS battery data:

• Convert battery status from 3 to 5 for full status
• Convert battery status from 1 (unplugged) to 3 (discharge)

Calculate battery episodes

• Only keep consecutive rows with charging (2) and discharging (3) events
• Put consecutive rows with the same battery status into the same group
• Calculate battery_level & timestamp differences between the first and the last row in each group respectively
• Discard rows with time difference <= 6 minutes to avoid including quick cycles

Extract battery features based on battery episodes

• Rows with battery_level difference > 0 are regarded as discharging episode
• Rows with battery_level difference <= 0 are regarded as charging episode
• Consumption rate is the ratio between battery_level difference and time difference for a discharging episode

We extract features for different day segments (epochs) from battery episodes: "countdischarge", "sumdurationdischarge", "countcharge", "sumdurationcharge", "avgconsumptionrate", "maxconsumptionrate".

[yiyir]

For battery preprocessing, why do we form raw episodes first, resample the episodes data by creating 2 duplicated rows for each minute bin, and then when extracting features we deduplicate(i.e.remove the second duplicated row for each bin), then form new episodes again using the raw episodes data(i.e. phone_battery_episodes.csv)? It seems that we created some redundant intermediate steps and reversed them before feature calculation?

Suggestion: Why don’t we form charging/discharging episodes from the beginning without the resampling in between? It seems that for battery feature calculation, we only form episodes based on: a) battery_status changes; b) time difference between consecutive rows exceeds episode_threshold_between_rows.

[JulioV]

Hi Yiyi

We resample any episode-based sensor (battery, activity recognition, screen, sleep intraday) because we need to fragment (chunk) any original episodes to fit the length of any requested time segments. For example:

Let's assume we are processing 15-minute FREQUENCY time segments:

ts1 00:00
ts2 00:15
ts3 00:30
ts4 00:45
ts5 01:00

And we have a battery discharge episode from 00:07 to 00:47, we should be able to fragment it into chunks:

00:07:00 - 00:14:59 belongs to ts1 
00:15:00 - 00:29:59 belongs to ts2 
00:30:00 - 00:44:59 belongs to ts3 
00:45:00 - 00:47:00 belongs to ts4

In essence, we divide every episode into the smallest mini-episodes that we need to handle (1 min), that we use to reconstruct new episodes within arbitrary time segments.

Doing it the other way around as you suggest might be feasible at first, for example assigning our battery episode 00:007-00:47 to ts1,ts2,ts3, and ts4 (we know the start/end timestamps for each ts so we could fragment the episode). However, it gets trickier when you consider the case when the episode spanned more than one time zone. Then we start doing the same DateTime/timezone processing logic in two places: when we tag the original battery episode and when we split it.

Similarly, we use the double row per 1-minute chunk to divide it at the second level otherwise we would need to resample in 1-second chunks. The first row is indexed by the episode start timestamp while the second row is indexed by the episode end timestamp. When the start and end rows belong to different time segments, we know we need to split the 1-minute chunk into smaller chunks. For example, this one-minute chunk of screen episode:

screen_sequence | start_timestamp | end_timestamp | start_date_time      | end_date_time
3,0             | 1588132750343   | 1588132810342 | 2020-04-28 23:59:10  | 2020-04-29 00:00:10

should be split into 2 parts if we are processing daily segments:

2020-04-28 23:59:10 to 2020-04-28 23:59:59 (50 seconds)
2020-04-29 00:00:00 to 2020-04-29 00:00:10 (10 seconds)

We considered that resampling and chunking would be better because it's a consistent process for all sensors. Every row spans one minute at max and it belongs to one time zone and to one or more time segments (done in src/data/datetime/readable_datetime.R). It also helps with auditing, since a user can see exactly how the episode is being split.

[yiyir]

Also, I'm not seeing this particular logic implemented in the code base - "Discard rows with time difference <= 6 minutes to avoid including quick cycles". I only saw logic related to episode_threshold_between_rows. Am I missing anything?

[JulioV]

I believe that condition was trying to patch a bug that we fixed last year. Since we changed the way we parse battery episodes I don't think it is needed anymore