Error with JSON data

[tlrjs]

In the upload function I am assigning a nested object to data. My code is as follows:

      $scope.upload[index] = $upload.upload(
        url: "/api/v1/resources"
        method: $scope.httpMethod
        headers:
          myHeaderKey: "myHeaderVal"
        data:
          resource: $scope.newResource

        file: $scope.selectedFiles[index]
        fileFormDataName: "file"

However, the parameters that are being sent to the server isn't being parsed correctly. For some reason the JSON data is being escaped. Shown below:

Parameters: {"resource"=>"{\"start_date\":\"02-05-2014\",\"tags\":[],\"end_date\":\"02-05-2015\",\"title\":\"test\",\"description\":\"test\"}", "file"=>#<ActionDispatch::Http::UploadedFile:0x007ff961f4c830 @tempfile=#<Tempfile:/var/folders/7h/m0tvrhd536qfxdqhpzct_tdc0000gn/T/RackMultipart20140205-1286-1xnpnrt>, @original_filename="crescent_desk.png", @content_type="image/png", @headers="Content-Disposition: form-data; name=\"file\"; filename=\"crescent_desk.png\"\r\nContent-Type: image/png\r\n">}

Any help is much appreciated. Thanks!

[tlrjs]

It seems to only be escaping the value ($scope.newResource) however the key (resource:) is parsed correctly along with the file information.

$scope.newResouce looks something like this:

{
  newResource:  
            { 
            start_date: 02-05-2014
            tags: []
            end_date: 02-05-2015
            title: test
            description: test
            }
 }

[danialfarid]

It applies the angular's default transformRequest which will call JSON.stringify() on your newResource.
If you want to customize the way they are added to FormData you can use formDataAppender option: #40 (comment)

Generally it is not a good design to send the data along with the file because the formData elements don't have a content-type and the data needs to be converted to plain text and back on the server side.
It is better to upload the file first then return an id and send the data along with that id to the server.

[tlrjs]

Interesting, I see now that it applies angular's default transformRequest if formDataAppender is not defined. However, when I do a standard $http.post with my $scope.newResource data (w/o the file) transformRequest is called and parses the data correctly.

So, I'm confused as to why transformRequest parses correctly on a standard angular $http.post request but not the directive.

[danialfarid]

Could you provide the actual content of the http request from the Chrome network tab.

[tlrjs]

Thanks for looking into this. Here is the content of the http request from the network tab when using the file upload directive. Let me know if you were asking for something else:

Request URL:http://localhost:9000/api/v1/resources
Request Method:POST
Status Code:500 Internal Server Error

Request Headersview source
Accept:application/json, text/plain, */*
Accept-Encoding:gzip,deflate,sdch
Accept-Language:en-US,en;q=0.8
Connection:keep-alive
Content-Length:8216
Content-Type:multipart/form-data; boundary=----WebKitFormBoundaryztZawy0EWfGyENWV
Host:localhost:9000
Origin:http://localhost:9000
Referer:http://localhost:9000/
User-Agent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.102 Safari/537.36
X-Requested-With:XMLHttpRequest

Request Payload
------WebKitFormBoundaryztZawy0EWfGyENWV
Content-Disposition: form-data; name="resource"

{"start_date":"02-05-2014","tags":[],"end_date":"02-05-2015","title":"test","description":"test"}
------WebKitFormBoundaryztZawy0EWfGyENWV
Content-Disposition: form-data; name="file"; filename="crescent_desk.png"
Content-Type: image/png


------WebKitFormBoundaryztZawy0EWfGyENWV--

Response Headersview source
connection:keep-alive
content-type:text/plain; charset=utf-8
Date:Thu, 06 Feb 2014 04:09:39 GMT
transfer-encoding:chunked
x-runtime:0.031392

Here is the content of the network tab when uploading just the form content using $http.post

Request URL:http://localhost:9000/api/v1/resources
Request Method:POST
Status Code:422 Unprocessable Entity

Request Headersview source
Accept:application/json, text/plain, */*
Accept-Encoding:gzip,deflate,sdch
Accept-Language:en-US,en;q=0.8
Connection:keep-alive
Content-Length:128
Content-Type:application/json;charset=UTF-8
Host:localhost:9000
Origin:http://localhost:9000
Referer:http://localhost:9000/
User-Agent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.102 Safari/537.36
X-Requested-With:XMLHttpRequest

Request Payloadview source
{,…}
resource: {start_date:02-05-2014, tags:[], end_date:02-05-2015, title:test title, description:test description}

Response Headersview source
cache-control:no-cache
connection:keep-alive
content-type:application/json; charset=utf-8
Date:Thu, 06 Feb 2014 04:21:46 GMT
set-
transfer-encoding:chunked
x-content-type-options:nosniff
x-frame-options:SAMEORIGIN
x-meta-request-version:0.2.8
x-runtime:0.026245
x-ua-compatible:chrome=1
x-xss-protection:1; mode=block

In the first one I have a 500 server error because the params are coming in with escaped quotes and the second one has a 422 unprocessable entity because I purposefully didn't send the file information which is required normally however I can see that the params aren't escaped in the second one.
Parameters: {"resource"=>{"start_date"=>"02-05-2014", "tags"=>nil, "end_date"=>"02-05-2015", "title"=>"test title", "description"=>"test description"}}
The main difference I noticed was the request payload looked different between the two.

[danialfarid]

You can see that your "resource" form data on the first one is in the correct json format so it might be something on your server side.
Be aware that you need to convert the string back to json on your server side using a json parser. It will be sent as string.
And as I said before you cannot set the content-type of the form data so that's why the second request look different because it can send it as "application/json" content type.

[tlrjs]

So it has to do with the content type? Because both are the same object and both are being sent through angular's transformRequest but one is received by the server with escaped quotes and the other is not.

[danialfarid]

They will both convert it to json string but since the content type is specified in post request the debugger and server side can show and receive it as a proper json object.

[wmbutler]

Well, I'm not really sure why this issue is closed. I'm having the same issue:

$upload.upload({
          method: 'POST',
          url: '/api/pdfSplit',
          headers: {'Content-Type': 'multipart/form-data' },
          // headers: {'Content-Type': $scope.files[index].type },
          file: $scope.files[index],
          data: { this : { id : "bill"} }
        })

but it's arriving in node express as:

{ this: '{"id":"bill"}' }

Based upon what I have read above, I guess I don't understand how to solve this issue. I need access to the JSON on the node side of things.

[wmbutler]

Alright,

I figured out the problem. When I originally set this up, I was trying to post from angular as:

$upload.upload({
          method: 'POST',
          url: '/api/pdfSplit',
          headers: {'Content-Type': 'multipart/form-data' },
          // headers: {'Content-Type': $scope.files[index].type },
          file: $scope.files[index],
          data: data
        })

This was causing a problem because sub-elements of data were getting wrapped in singlequotes. My solution was to do the following:

$upload.upload({
          method: 'POST',
          url: '/api/pdfSplit',
          headers: {'Content-Type': 'multipart/form-data' },
          // headers: {'Content-Type': $scope.files[index].type },
          file: $scope.files[index],
          data: { "foo" : data }
        })

Now, I can access it from express as:

req.body.foo

[matthoiland]

@wmbutler When using your method I still can't see my data in Express.

Here's my Angular

$upload.upload({
     url: '/upload',
    method: 'POST',
    headers: {'Content-Type': 'multipart/form-data'},
    file: file,
    data: { 
        "crop": {
            "regular": "400x200",
            "full": "800x1200"
        }
    }
})

And here's my Express

app.post('/upload', function(req, res){
    console.log(req.body.crop);
    res.send('hello');
});

Here's what I see in my Chrome network panel:

Content-Disposition: form-data; name="crop"
{"regular":"400x200","full":"800x1200"}

I'm using body-parser and connect-busboy and can't get this data.

[wmbutler]

I'd suggest a little housekeeping. It's the only way I could keep things straight. I hope this works for you. First, assign the contents of your JSON to a variable:

stuff = { 
            "regular": "400x200",
            "full": "800x1200"
        };

$upload.upload({
     url: '/upload',
    method: 'POST',
    headers: {'Content-Type': 'multipart/form-data'},
    file: file,
    data: { "crop" : stuff }
})

Then, I had to JSON.Parse on the express side

var data = JSON.parse(req.body.crop);

Let me know how it goes. I wrestled quite a bit with this.

[mdurn]

I found this thread while I was struggling with the same issue. However, I found that my problem was that I was misusing the concept of form data. For those still struggling, this is what I did:

Say I have some data that I want to post:

var data = { 'stuff': { 'foo': 'bar' } }

My initial (wrong) thought was to just use this variable as the data in my upload function:

$upload.upload({
  url: "/upload",
  method: 'POST',
  data: data,
  file: file
})

That's not going to work and ends up having the same problem people on this thread are talking about, where on the server I receive a string for the value:

{"stuff"=>"{\"foo\":\"bar\"}"

We instead have to use the formDataAppender option to format the form data correctly:

$upload.upload({
  url: "/upload",
  method: 'POST',
  data: data,
  file: file,
  formDataAppender: function(formData, key, value){
    // key => 'stuff'
    // value => { 'foo': 'bar' }
    for (k in value) {
      var dataKey = key + "[" + k + "]"; // => "stuff[foo]"
      formData.append(dataKey, value[k]); // => sets "stuff[foo]=bar" - the correct way to send the form data
    }
  }
})

I know this is a closed issue, but hopefully that helps anyone who is having the same issue and finds this thread.

[juninhoall]

Your problem is in.
var data = { 'stuff': { 'foo': 'bar' } }

import library jquery todo parsing

$.param(data);

is solution your problems.