Enhance appsync dynamodb example with multiple table relationships

[sid88in]

No description provided.

[eltonio450]

Hi :),

Finally successfully implemented with dynamodb + mapping templates. Unfortunately I don't have time for a full PR, but I post small explanation here (since it's been a pain to find something related to this on the AWS doc)

The key is to link mapping templates through the "source" argument. See here: https://docs.aws.amazon.com/appsync/latest/devguide/resolver-context-reference.html#dynamodb-helpers-in-util-dynamodb

In my case, I have an Image, which contains Objects, with two distinct dynamoDB Tables. I would like to query something like this:

query imagesList ($uploadBatch: String!, $nextToken: String){
        imagesList(uploadBatch: $uploadBatch, limit: 5, nextToken: $nextToken) @connection(key: "imagesList"){
            items {
                id
                name
                hash
                path
                objects {
                    id
                    value
                }
            }
            nextToken
        }

where id, name, hash and path are stored in one table, and objects in another

My YAML looks like this:

mappingTemplates:
      - dataSource: Objects
        type: Image
        field: objects
        request: "root/infos-request-mapping-template.txt"
        response: "root/infos-response-mapping-template.txt"
      - dataSource: Images
        type: Query
        field: imagesList
        request: "image/imagesList-request-mapping-template.txt"
        response: "image/imagesList-response-mapping-template.txt"
    schema: schema.graphql
    serviceRole: "AppSyncServiceRole-${opt:stage}"
    dataSources:
      - type: AMAZON_DYNAMODB
        name: Images
        description: 'Image table'
        config:
          tableName: 'Images-${opt:stage}'
          serviceRoleArn: "arn:aws:iam::${self:custom.accountId}:role/Dynamo-${self:custom.appSync.serviceRole}"
      - type: AMAZON_DYNAMODB
        name: Objects
        description: 'Objects table'
        config:
          tableName: 'Objects-${opt:stage}'
          serviceRoleArn: "arn:aws:iam::${self:custom.accountId}:role/Dynamo-${self:custom.appSync.serviceRole}"

To be noted: the primary key for an object in the Objects table is the same as the primary key of the image. However, this is not necessary for this example.

My mapping templates look like this:

getImagesList:

{
    "version" : "2017-02-28",
    "operation" : "Query",
    "query" : {
        ## Provide a query expression. **
        "expression": "#owner = :owner",
        "expressionNames" : { "#owner": "owner"},
        "expressionValues" : {
            ":owner" : {
                "S" : "${context.identity.username}"
            }
        }
    },
    ## Add 'limit' and 'nextToken' arguments to this field in your schema to implement pagination. **
    "limit": $util.defaultIfNull(${ctx.args.limit}, 10),
    "index": "owner-index",
    "nextToken": $util.toJson($util.defaultIfNullOrBlank($ctx.args.nextToken, null))
}

and infos:

    "version" : "2017-02-28",
    "operation" : "Query",
    "query" : {
        ## Provide a query expression. **
        "expression": "image = :image",
        "expressionValues" : {
            ":image" : {
                "S" : "${ctx.source.id}"
            }
        }
    }
}

The key point is 'source' :). Hope it helps, it took me an entire week to understand how to get parents arguments :) !

PS: there is another example of source in 'topTweet-request-mapping-template.txt'

[sid88in]

@eltonio450 thanks a lot! I'll take a look sometime this week.

[Zhuravlev1]

@sid88in @eltonio450 Hi, I have same problem. I have two tables. User and Post and I need get post and information about author. User.userId = Post.userId. Could you help me ?
The schema, mapping template and Query I created are as follows:

type PostConnection {
	items: [UserPost]
	nextToken: String
}

type Query {
	# get post
	listUserPosts(userId: ID!, limit: Int, nextToken: String): PostConnection
	# Get a single profile by user id
	getStorefrontProfile(userId: ID!): UserProfile
}

type UserPost {
	userId: ID!
	createdAt: String!
	content: String!
	author: [UserProfile]
}

type UserProfile {
	userId: ID!
	name: String!
	profileImage: String!
}

// mapping template
{
  "version": "2017-02-28",
  "operation": "Query",
  "query" : {
  "expression": "userId = :id",
    "expressionValues" : {
      ":id" : {
        "S" : "${ctx.source.userId}"
      }
    }
  },
  "scanIndexForward": false,
  "limit": $util.defaultIfNull($ctx.args.limit, 20),
  "nextToken": $util.toJson($util.defaultIfNullOrEmpty($ctx.args.nextToken, null))
}

//query
query {
    listUserPosts (userId: "myUserId") {
    items {
    	content
      	author {
	        userId
	        name
	        profileImage
      }
    }
  }
}

and I have next response:

{
  "data": {
    "listUserPosts": {
      "items": []
    }
  }
}

but if use "${ctx.args.userId}" instead "${ctx.source.userId}" I have

"items": [{
  "userId": "myUserId",
  "content": "Hello, world!",
  "user": null
}]

[eltonio450]

Hi,

First, you need to slightly modify your request to include userId on the 'items' level (which then will be passed in 'source' to the next level):

query {
    listUserPosts (userId: "myUserId") {
    items {
    	content
        userId
      	author {
	        userId
	        name
	        profileImage
      }
    }
  }
}

Then, you should have two mapping templates:

One which resolves 'items' in PostConnection (or directly listUserPosts), and should look like this (not tested, modified yours), with Post as Datasource.

{
  "version": "2017-02-28",
  "operation": "Query",
  "query" : {
  "expression": "userId = :id",
    "expressionValues" : {
      ":id" : {
        "S" : "${context.arguments.userId}"
      }
    }
  },
  "scanIndexForward": false,
  "limit": $util.defaultIfNull($ctx.args.limit, 20),
  "nextToken": $util.toJson($util.defaultIfNullOrEmpty($ctx.args.nextToken, null))
}

At this point, you collected all the posts given a userId

You now need to resolve the field 'author' for every item. The mapping template, with User as Datasource, will look like this (if you don't have a sort key on your Users table):

{
    "version" : "2017-02-28",
    "operation" : "GetItem",
    "key" : {
        "id" : { "S" : "${context.source.userId}" }
    }
}

(btw you can batchInvoke here)

Hope it helps

[ashish-nodejs]

Hi i have some issue in app sync api
my schema is like
// SCHEMA

type Query {
GetUser(UserId: ID!): User
CheckUserName(Username: String): User
}

type User {
UserId: ID!
UName: String
Username: String
CreatedOn: Int
}

schema {
query: Query
}

// QUERY
query CheckUserName{
CheckUserName (Username: "test_user") {
UserId
UName
Username
}
}

and my Request Mapping Template Code
{
"version" : "2017-02-28",
"operation" : "DeleteItem",
"key": {
"Username": { "S" : "${context.arguments.Username}"}
}
}
#######
{
"data": {
"CheckUserName": null
},
"errors": [
{
"path": [
"CheckUserName"
],
"data": null,
"errorType": "DynamoDB:AmazonDynamoDBException",
"errorInfo": null,
"locations": [
{
"line": 2,
"column": 5,
"sourceName": null
}
],
"message": "The provided key element does not match the schema (Service: AmazonDynamoDBv2; Status Code: 400; Error Code: ValidationException; Request ID: F216ABF39BG68N5TS1POEG4JP7VV4KQNSO5AEMVJF66Q9ASUAAJG)"
}
]
}

[harleyguru]

Hi @eltonio450

{
    "version" : "2017-02-28",
    "operation" : "GetItem",
    "key" : {
        "id" : { "S" : "${context.source.userId}" }
    }
}

Here, if $context.source.userId will be null or empty, what about?
This GetItem template won't work.
What will you suggest to handle this case?
Thanks

[bmccann36]

In response to anyone trying to query two different Dynamo tables to access related data:
Rather than trying to design your graphQl api to do this you may want to take a step back and redesign your tables + data modeling strategy.
From the AWS dynamoDB documentation:
"You should maintain as few tables as possible in a DynamoDB application. Most well designed applications require only one table" https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/bp-general-nosql-design.html
For people coming from an RDS background it can be hard to wrap your head around. However when you think about it, and actually start building an app that requires multiple tables for all of your related entities you may realize that implementing dynamo the same way you would implement an RDS presents some serious problems. When you use RDMS you usually work with some type of ORM that helps you with things like 1) retrieving related items across tables with one api call 2) cascading updates + deletes across tables etc...
DynamoDb in contrast does not offer these things and there is a reason for that. It is not meant to be used in this way.
Using the scenario in the example above you could design a table that holds authors and posts
partition key------sort key-----other attributes
---userId:01--------postId:01 ..............post one content
---userId:01--------postId:02 . ............post two content
---userId:01--------userName:Bob ........other data about user

In this manner we are using partitions effectively. DynamoDB will actually store items with the same partition key physically close together in memory. Now when we want to get post data as well as data about a user we can do it in one query. If we just want posts and no user data we make the following query: PK=userId:1, SK begins_with(postId). I think it makes the query a lot simpler too. Taking this pattern even further, having Id's becomes kind of pointless. Instead of having post Id's we can throw that out and use timestamps instead. Now we can execute queries like partitionKey=01 and sortKey greater_than() will give us all a user's posts within a time range.

This will be a lot faster, cheaper, and maintainable than provisioning and querying two separate tables.
I am not saying this is the correct approach for every application but in many scenarios it makes sense ,like for building an e-commerce site (such as Amazon).

good talk on this topic here
https://www.youtube.com/watch?v=jzeKPKpucS0

[planetDark]

Is there a way in Appsync to write a recursive function.. Something like iterate over a loop of record gratear than 1 MB from the data source in appsync , process all the records and then push it to client ?