Skip to content

Conversation

@laresbernardo
Copy link
Contributor

As described in #1279

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jun 12, 2025
@laresbernardo
Copy link
Contributor Author

Validated matching values across different approaches and sources. Any model should work to replicate.

library(dplyr)

start_date <- min(InputCollect$dt_mod$ds)
end_date <- max(InputCollect$dt_mod$ds)

# Gather data needed
dt_mod <- select(InputCollect$dt_mod, c("ds", InputCollect$all_media)) %>%
  filter(ds >= start_date, ds <= end_date)
periods <- nrow(dt_mod)
df <- filter(OutputCollect$mediaVecCollect, solID %in% select_model, .data$type == "decompMedia") %>%
  filter(ds >= start_date, ds <= end_date)

# Mean response for total modeling window
(mean_resp <-  select(df, InputCollect$paid_media_spends) %>%
  summarise_all(function(x) sum(x, na.rm = TRUE)/periods))
# Mean spend for total modeling window
(mean_spends <- select(dt_mod, InputCollect$paid_media_spends) %>%
  summarise_all(function(x) sum(x, na.rm = TRUE)/periods))
# Mean performance
(perf <- mean_resp/mean_spends)

# Check performance values from one-pager (match!)
op <- robyn_onepagers(InputCollect, OutputCollect, select_model, export = FALSE)
op[[1]][[2]][[1]]$layers[[3]]$data %>% select(rn, value)

# Check values from lares::robyn_performance() (match!)
lares::robyn_performance(InputCollect, OutputCollect, solID = select_model) %>%
  filter(type == "Paid") %>%
  select(channel, performance, spend, response) %>%
  mutate(response = response / periods,
         spend = spend / periods)

# Check match with robyn_write() -> fixed in fix_ba_perf branch
(exp <- robyn_write(
  InputCollect, OutputCollect,
  select_model = select_model, export = FALSE)$ExportedModel$summary %>%
    select(variable, mean_response, mean_spend, performance) %>%
    filter(!is.na(performance)))

# Different source, same results (xDecompAgg & mediaVecCollect:decompMedia - match)
(df2 <- filter(OutputCollect$xDecompAgg, .data$solID == select_model) %>%
    filter(.data$rn %in% InputCollect$paid_media_spends) %>%
    group_by(.data$solID, .data$rn) %>%
    summarise(spend = sum(.data$total_spend),
              response = sum(.data$xDecompAgg),
              mean_spend = .data$spend / periods,
              mean_resp = .data$response / periods,
              performance = sum(.data$xDecompAgg) / sum(.data$total_spend),
              .groups = "drop"))

### NOW, BUDGET ALLOCATOR

# Spend matches, response vales were very far off, and in consequence, performance off - fixed!
select(AllocatorCollect1$dt_optimOut, initResponseUnit, initSpendUnit, initRoiUnit)

@laresbernardo laresbernardo requested a review from gufengzhou June 12, 2025 01:21
@laresbernardo laresbernardo self-assigned this Jun 12, 2025
Copy link
Contributor

@gufengzhou gufengzhou left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@laresbernardo laresbernardo merged commit ac773d7 into main Jun 12, 2025
1 check passed
@laresbernardo laresbernardo deleted the fix_ba_perf branch June 12, 2025 23:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants