req_body_multipart fails when one named element is a list

[GreenGrassBlueOcean]

I am writing an r package that will work with https://github.com/bbernhard/signal-cli-rest-api
so that one can send messages using Signal. I am planning to make this open source once finished.

To send a message to (multiple) recipients the api expects a list of multiple character numbers.
This works fine as long as I don't add an Base 64 attachment:

Without attachment works (and I also receive the message in signal when I replace req_dry_run with req_perform)

library(httr2)

# curl -X POST -H "Content-Type: application/json" -d '{"message": "<message>", "number": "<number>",
# "recipients": ["<recipient1>", "<recipient2>"]}' 'http://127.0.0.1:8080/v2/send'

Request_url <- "http://127.0.0.1:8080/v2/send"

Request_body <-  list( 'message' = "Hello World!"
                     , 'number' =  "+1123456789"
                     , "recipients" = list("+1987654321", "+1987654322")
                     )

query <- httr2::request(base_url = Request_url) |>
  httr2::req_headers('Content-Type' = 'application/json') |>
  httr2::req_timeout(60) |>
  httr2::req_body_json(Request_body)|>
  httr2::req_dry_run()
#> POST /v2/send HTTP/1.1
#> Host: 127.0.0.1:8080
#> User-Agent: httr2/1.0.0 r-curl/5.2.0 libcurl/8.3.0
#> Accept: */*
#> Accept-Encoding: deflate, gzip
#> Content-Type: application/json
#> Content-Length: 92
#> 
#> {"message":"Hello World!","number":"+1123456789","recipients":["+1987654321","+1987654322"]}

^{Created on 2024-03-06 with reprex v2.1.0}

With attachment it does not work anymore because it wants that recipients should be a named list.
However this is not supported by the api (when I remove the list, it will return a 400 error from the api)

library(httr2)
library(RCurl)
library(ggplot2)

# curl -X POST -H "Content-Type: application/json" -d '{"message": "<message>"
#, "base64_attachments": ["<base64 encoded attachment>"]
# , "number": "<number>", "recipients": ["<recipient1>", "<recipient2>"]}' 'http://127.0.0.1:8080/v2/send'

p <- ggplot(mtcars, aes(wt, mpg)) + geom_point() 
ggsave(p, filename = "test.png")
base64_attachments <- "test.png"

Request_body <-  list( 'message' = "Hello World!"
                       , 'number' =  "+1123456789"
                       , "recipients" = list("+1987654321", "+1987654322")
)

query <- httr2::request(base_url = Request_url) |>
  httr2::req_headers('Content-Type' = 'application/json') |>
  httr2::req_user_agent("RSignalApi") |>
  httr2::req_timeout(60) |>
  httr2::req_body_json(Request_body ) |>
  httr2::req_body_multipart(base64_attachments = curl::form_file(base64_attachments))

query
#> <httr2_request>
#> POST http://127.0.0.1:8080/v2/send
#> Headers:
#> • Content-Type: 'application/json'
#> Body: multipart encoded data
#> Options:
#> • useragent: 'RSignalApi'
#> • timeout_ms: 60000
query$body
#> $data
#> $data$message
#> [1] "Hello World!"
#> 
#> $data$number
#> [1] "+1123456789"
#> 
#> $data$recipients
#> $data$recipients[[1]]
#> [1] "+1987654321"
#> 
#> $data$recipients[[2]]
#> [1] "+1987654322"
#> 
#> 
#> $data$base64_attachments
#> Form file: test.png 
#> 
#> 
#> $type
#> [1] "multipart"
#> 
#> $content_type
#> NULL
#> 
#> $params
#> list()

query |>  httr2::req_dry_run()
#> Error in curl::handle_setform(handle, .list = req$fields): Unsupported value type for form field 'recipients'.

^{Created on 2024-03-06 with reprex v2.1.0}

[hadley]

Somewhat more minimal reprex:

library(httr2)

path <- tempfile()

png(path)
plot(1:10)
dev.off()
#> quartz_off_screen 
#>                 2

body <- list(
  'message' = "Hello World!",
  'number' =  "+1123456789",
  "recipients" = list("+1987654321", "+1987654322")
)

request("http://127.0.0.1:8080/v2/send") |>
  req_body_json(body) |>
  req_dry_run(quiet = TRUE)

request("http://127.0.0.1:8080/v2/send") |>
  req_body_json(body) |>
  req_body_multipart(base64_attachments = curl::form_file(path)) |> 
  req_dry_run(quiet = TRUE)
#> Error in curl::handle_setform(handle, .list = req$fields): Unsupported value type for form field 'recipients'.

^{Created on 2024-03-07 with reprex v2.1.0}

[hadley]

Oh hmmm, the problem is really that req_body_json() and req_body_multipart() are mutually exclusive, but clearly something is going wrong when flipping from one to the other. I'll definitely fix that, but it won't really help your problem because it will just cause the request creation to error. Do you have more details about the request you are trying to create (i.e. a link to the docs) so I can think about how you should describe the request with httr2?

[GreenGrassBlueOcean]

Hi Hadley,

Many thanks for your response and I am sorry that I missed your second post somehow.
I was able to make a workaround by changing the unnamed lists into a vector.
I now duplicate the vector when the vector has a length of one.

Downside is that I now will receive the same attachment two times in case of a single attachment.
Which is not ideal but better than no attachment.

Otherwise the generated curl command will not contain the square brackets for recipients and also attachments:

"recipients": ["<recipient1>", "<recipient2>"]'

The unit tests still need to be written using the r package webfakes.
For the documentation I have already written several examples with special characters and various type of attachments and sofar it looks ok.

The requested documentation can be found here:
Github repo Signal CLI rest api

curl coded examples can be found here:
https://github.com/bbernhard/signal-cli-rest-api/blob/master/doc/EXAMPLES.md

the example for sending a message with a base 64 example is:
`e.g: TMPFILE="$(base64 image_9.jpg)"
curl -X POST -H "Content-Type: application/json" -d '{"message": "Test image", "base64_attachments": ["'"${TMPFILE}"'"], "number": "+431212131491291", "recipients": ["+4354546464654"]}' 'http://127.0.0.1:8080/v2/send'

I wrote the following implementation for r.

SendMessage <- function( message = NULL, number = Sys.getenv(".SignalServerPhoneNumber")
                       , recipients = NULL, attachment_path = NULL){

  if(is.null(message)){
    stop("Message cannot be NULL")
  }

   if(is.null(recipients) | !is.vector(recipients)){
     stop("recipients can  not be NULL and has to be a vector")
   }

  if(length(recipients) == 1){
    recipients <- rep_len(recipients, length.out = 2)
  }

 # Here I create a single list with all parameters for the message
  json <- list( 'message' = message
              , 'number' = number
              , "recipients" = recipients
              , "base64_attachments" = attachment_path
              )
  json[sapply(json, is.null)] <- NULL

  query <- ExecuteRequest( Request_url = paste0(GetBaseUrl(),"v2/send")
                                           , JsonBody = json
                                           , request_type = "POST"
                                           )

  return(query)

}

^{Created on 2024-03-12 with reprex v2.1.0}

ExecuteRequest <- function(Request_url = NULL, JsonBody = NULL, request_type = NULL){

  if(toupper(request_type) == "POST"){

    if(any(is.null(Request_url), is.null(JsonBody))){
      stop("Request_url or JsonBody cannot be null")
    }

    if("base64_attachments" %in% names(JsonBody)){
      encode_file_to_base64 <- function(file_path) {
        file_content <- RCurl::base64Encode(readBin(file_path, "raw", file.info(file_path)$size))
        return(as.character(file_content))
    }

    if(length(JsonBody$base64_attachments)==1L){
        JsonBody$base64_attachments <- rep_len(JsonBody$base64_attachments, length.out = 2)
    }

    JsonBody$base64_attachments <- lapply( X = JsonBody$base64_attachments
                                         , FUN = encode_file_to_base64
                                         ) |> unlist()

    }

    query <- httr2::request(base_url = Request_url) |>
             httr2::req_headers('Content-Type' = 'application/json') |>
             httr2::req_user_agent("RSignalApi") |>
             httr2::req_timeout(60) |>
             httr2::req_body_json(JsonBody) |>
             httr2::req_perform()

  } else {
    stop("not implemented request_type")
  }
 
  return(query)
}

^{Created on 2024-03-12 with reprex v2.1.0}

As can be observed from above I would personally like to be able to send a request using req_body_json that contains base64 files without the added complexity of choosing if I have a req_body_multipart or not.

I guess that in almost all cases in which attachments are send other data is in the same body of the POST message.